Integration Tests

Integration tests are separated into multi modules. Each module requires a different state and configuration. These configurations are managed externally. An overview of the entire layered setup of different integration test components is shown below

To minimize runtime overhead and memory pressure it is best to execute the dummy integration test module before any other container is created. This will ensure pytango is installed in the .tox/integration virtual environment before the system experiences the severe memory pressure of the ELK stack.

Simply run make integration dummy from inside the docker-compose folder.

In total the orchestration of integration tests is handled through four separate layers, each one calling the next:

1. sbin/run_integration_test.sh
2. docker-compose/Makefile - integration target
3. docker-compose/integration-test.yml
4. tangostrationcontrol/tox.ini - integration job

Invocation of individual modules is achieved through tox, this environment value can be left empty for the default module: TEST_MODULE=modulefolder tox -e integration

Individual tests can be invoked using arguments:

TEST_MODULE=default tox -e integration import.path.class.functionname

These arguments and modules can also be passed at the level of the Makefile instead of through tox directly:

make integration default import.path.class.functionname`

Since at each layer tests can be specified, the information amount selection modules and tests has to be passed down from the top layers to the ones below. To be able to understand the flow of this information the following diagram below is provided.

It should be noted that not all layers of the setup are convenient to interact with. Some should be avoided see diagram below.

Breakpoints & Debuggers with Integration Tests

It can be extremely helpful to be able to mount a debugger during integration tests. This can be achieved in two or three steps depending on the availability of tango on the host

1. docker exec into a container based on lofar-device-base
2. Navigate to the tangostationcontrol directory
3. Execute tox -e integration to generate the virtual environment
4. Activate the virtual environment source .tox/testenv/bin/activate
5. Modify the source files and add import pdb; pdb.set_trace() where you want breakpoints.
6. Execute the desired test using testtools python -m testtools.run imort.class.path.functionname

A concrete example is shown below:

docker exec -it device-sdp /bin/bash
cd tangostationcontrol
# Single test to significantly reduce runtime
tox -e integration -- tangostationcontrol.integration_test.default.devices.test_device_digitalbeam.TestDeviceDigitalBeam.test_pointing_to_zenith
source .tox/testenv/bin/activate
# Add import pdb; pdb.set_trace() somewhere
nano integration -- tangostationcontrol.integration_test.default.devices.test_device_digitalbeam.py
# If the test is in the default module
python -m testtools.run tangostationcontrol.integration_test.default.devices.test_device_digitalbeam.TestDeviceDigitalBeam.test_pointing_to_zenith
# If the test is in the configuration module
TEST_MODULE=configuration python -m testtools.run tangostationcontrol.integration_test.configuration.test_device_configuration.TestDeviceConfiguration.test_read_station_configuration

Approach

A special docker container is build to perform the integration tests. This container will be build by the makefiles but should only be started by the dedicated integration test script. This script will ensure that other containers are running and are in the required state.

• Launch simulators.
• Reconfigure dsconfig to use these simulators.
• Create and start the integration-test container for the specific module.

Running

Warning running these tests will make changes to your CDB database config!

sbin/run_integration_test.sh

Cleanup, recovery or starting over

All docker content including images, containers, networks and volumes can be deleted using the following:

docker stop $(docker ps | tail -n+2 | awk '{NF=1}1' | awk '{printf("%s ",$0)} END { printf "\n" }'); docker system prune --all; docker volume prune