README.md

# Unit Testing

Is the procedure of testing individual units of code for fit for use. Often
this entails isolating specific segments and testing the behavior given a
certain range of different inputs. Arguably, the most effective forms of unit
testing  deliberately uses _edge cases_ and makes assumptions about any other
sections of code that are not currently under test.

First the underlying technologies as well as how they are applied in Tango
Station Control are discussed. If this seems uninteresting, or you are already
familiar with these concepts you can skip to
[Running tox tasks](#running tox tasks) and
[Debugging unit tests](#debugging-unit-tests).

### Table of Contents:

- [Tox](#tox)
- [Testing approach](#testing-approach)
- [Mocking](#mocking)
- [Running tox tasks](#running-tox-tasks)
- [Debugging unit tests](#debugging-unit-tests)

## Tox

[Tox](https://tox.readthedocs.io/en/latest/) is a commandline tool to simplify
running Python tasks such as linting and unit testing. Using a simple
[configuration file](../tox.ini) it can setup a
[virtual environment](https://virtualenv.pypa.io/en/latest/) that automatically
installs dependencies before executing the task. These environments persist
after the task finishes preventing the excessive downloading and installation
of dependencies.

The Tox environments in this project are configured to install any dependencies
listed in [test-requirements.txt](../test-requirements.txt) and
[lofar-requirements.txt](../../docker-compose/lofar-device-base/lofar-requirements.txt)
this can also  easily be verified within our [configuration file](../tox.ini).

## Testing approach

For testing [stestr](https://stestr.readthedocs.io/en/latest/) is used this
tool  has main advantage the utilization of a highly parallelized unit test
runner that automatically scales to the number of simultaneous threads
supported by the host processor. Other features include automatic test
discovery and pattern matching to only execute a subset of tests.

However, stestr is incompatible with using breakpoints
(through [pdb](https://docs.python.org/3/library/pdb.html)) directly and  will
simply fail the test upon encountering one. The
[debugging unit tests](#debugging-unit-tests) section describes how to mitigate
this and still debug individual unit tests effectively.

All tests can be found in this test folder and all tests must inherit from
`TestCase`, found in [base.py](base.py). This ensures that any test fixtures run
before and after tests. These test fixtures allow to restore any global modified
state, such as those of static variables. It is the task of the developer
writing the unit test to ensure that any global modification is restored.

When writing unit tests it is best practice to mimic the directory structure of
the original project inside the test directory. Similarly, copying the filenames
and adding _test__ to the beginning. Below is an example:

* root
  * database
    * database_manager.py
  * test
    * base.py
    * database
        * test_database_manager.py

## Mocking

Contrary to many other programming languages, it is entirely possible to
modify **any** function, object, file, import at runtime. This allows for a
great deal of flexibility but also simplicity in the case of unit tests.

Isolating functions is as simple as mocking any of the classes or functions it
uses and modifying its return values such as specific behavior can be tested.
Below is a simple demonstration mocking the return value of a function using the
mock decorator. For more detailed explanations see
[the official documentation](https://docs.python.org/3/library/unittest.mock.html).

```python
from unittest import mock

# We pretend that `our_custom_module` contains a function `special_char`
import our_custom_module

def function_under_test(self):
    if our_custom_module.special_char():
        return 12
    else:
        return 0

@mock.patch.object(our_custom_module, "special_char")
def test_function_under_test(self, m_special_char):
    """ Test functionality of `function_under_test`

    This mock decorator _temporarily_ overwrites the :py:func:`special_char`
    function within :py:module:`our_custom_module`. We get access to this
    mocked object as function argument. Concurrent dependencies of these mock
    statements are automatically solved by stestr.
    """

    # Mock the return value of the mocked object to be None
    m_special_char.return_value = None

    # Assert that function_under_test returns 0 when special_char returns None.
    self.assertEqual(0, function_under_test())

```

## Running tox tasks

Running tasks defined in Tox might be a little different than what you are used
to. This is due to the Tango devices running from Docker containers. Typically,
the dependencies are only available inside Docker and not on the host.

The tasks can thus only be accessed by executing Tox from within a Tango device
Docker container. A simple interactive Docker exec is enough to access them:

```sh
docker exec -it device-sdp /bin/bash
cd /opt/lofar2.0/tango/devices/
tox
```

For specific tasks the `-e` parameter can be used, in addition, any arguments
can be appended after the tasks. These arguments can be interpreted within the
Tox  configuration file as `{posargs}`. Below are a few examples:

```sh
# Execute unit tests with Python 3.7 and only execute the test_get_version test
# from the TestLofarGit class found within util/test_lofar.py
# Noteworthy, this will also execute test_get_version_tag_dirty due to pattern
# matching.
tox -e py37 util.test_lofar.TestLofarGit.test_get_version_tag
# Execute linting
tox -e pep8
```

## Debugging unit tests

Debugging works by utilizing the
[virtual environment](https://virtualenv.pypa.io/en/latest/)that Tox creates.
These are placed in the .tox/ directory. Each of these environments carries
the same name as found in _tox.ini_, these match the names used for `-e`
arguments

Debugging unit tests is done by inserting the following code segment just before
where you think issues occur:

```python
import pdb; pdb.set_trace()
```

Now as said stestr will catch any breakpoints and reraise them so we need to
avoid using stestr while debugging. Simply source the virtual environment
created by tox `source .tox/py37/bin/activate`. You should now see that the
shell $PS1 prompt is modified to indicate the environment is active.

From here execute `python -m testtools.run` and optionally the specific test
case as command line argument. These test will not run in parallel but support
all other features such as autodiscovery, test fixtures and mocking.

Any breakpoint will be triggered and you can use the pdb interface
(very similar to gdb) to step through the code, modify and print variables.

Afterwards simply execute `deactivate` to deactivate the virtual environment.
**DO NOT FORGOT TO REMOVE YOUR `pdb.set_trace()` STATEMENTS AFTERWARDS**

The best approach to prevent committing `import pdb; pdb.set_trace()` is to
ensure that all unit tests succeed beforehand.