diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..b7b4398a9581bf0771fa2e8a669f1e53c92b75d2 --- /dev/null +++ b/README.md @@ -0,0 +1,3 @@ +# Tango Station Control + +Station Control software related to Tango devices. \ No newline at end of file diff --git a/devices/README.md b/devices/README.md index 1d962a271e60cc125feb88d5b1ce1b7c5f3d627b..0604c3a4d1d90b7d99ab1f35cee1922c6ee99371 100644 --- a/devices/README.md +++ b/devices/README.md @@ -1,4 +1,4 @@ -# Device wrapper +# Tango Station Control Device wrappers This code provides an attribute_wrapper class in place of attributes for tango devices. the attribute wrappers contain additional code that moves a lot of the complexity and redundant code to the background. diff --git a/devices/test/README.md b/devices/test/README.md new file mode 100644 index 0000000000000000000000000000000000000000..549b2ddedd2121483f66ce1324f200088dbe4dcb --- /dev/null +++ b/devices/test/README.md @@ -0,0 +1,172 @@ +# Unit Testing + +Is the procedure of testing individual units of code for fit for use. Often +this entails isolating specific segments and testing the behavior given a +certain range of different inputs. Arguably, the most effective forms of unit +testing deliberately uses _edge cases_ and makes assumptions about any other +sections of code that are not currently under test. + +First the underlying technologies as well as how they are applied in Tango +Station Control are discussed. If this seems uninteresting, or you are already +familiar with these concepts you can skip to +[Running tox tasks](#running tox tasks) and +[Debugging unit tests](#debugging-unit-tests). + +###Table of Contents: + +- [Tox](#tox) +- [Testing approach](#testing-approach) +- [Mocking](#mocking) +- [Running tox tasks](#running-tox-tasks) +- [Debugging unit tests](#debugging-unit-tests) + +## Tox + +[Tox](https://tox.readthedocs.io/en/latest/) is a commandline tool to simplify +running Python tasks such as linting and unit testing. Using a simple +[configuration file](../tox.ini) it can setup a +[virtual environment](https://virtualenv.pypa.io/en/latest/) that automatically +installs dependencies before executing the task. These environments persist +after the task finishes preventing the excessive downloading and installation +of dependencies. + +The Tox environments in this project are configured to install any dependencies +listed in [test-requirements.txt](../test-requirements.txt) and +[lofar-requirements.txt](../../docker-compose/lofar-device-base/lofar-requirements.txt) +this can also easily be verified within our [configuration file](../tox.ini). + +## Testing approach + +For testing [stestr](https://stestr.readthedocs.io/en/latest/) is used this +tool has main advantage the utilization of a highly parallelized unit test +runner that automatically scales to the number of simultaneous threads +supported by the host processor. Other features include automatic test +discovery and pattern matching to only execute a subset of tests. + +However, stestr is incompatible with using breakpoints +(through [pdb](https://docs.python.org/3/library/pdb.html)) directly and will +simply fail the test upon encountering one. The +[debugging unit tests](#debugging-unit-tests) section describes how to mitigate +this and still debug individual unit tests effectively. + +All tests can be found in this test folder and all tests must inherit from +`TestCase`, found in [base.py](base.py). This ensures that any test fixtures run +before and after tests. These test fixtures allow to restore any global modified +state, such as those of static variables. It is the task of the developer +writing the unit test to ensure that any global modification is restored. + +When writing unit tests it is best practice to mimic the directory structure of +the original project inside the test directory. Similarly, copying the filenames +and adding _test__ to the beginning. Below is an example: + +* root + * database + * database_manager.py + * test + * base.py + * database + * test_database_manager.py + +## Mocking + +Contrarily to many other programming languages, it is entirely possible to +modify **any** function, object, file, import at runtime. This allows for a +great of flexibility but also simplicity in the case of unit tests. + +Isolating functions is as simple as mocking any of the classes or functions it +uses and modifying its return values such as specific behavior can be tested. +Below is a simple demonstration mocking the return value of a function using the +mock decorator. For more detailed explanations see +[the official documentation](https://docs.python.org/3/library/unittest.mock.html). + +```python +from unittest import mock + +# We pretend that `our_custom_module` contains a function `special_char` +import our_custom_module + +def function_under_test(self): + if our_custom_module.special_char(): + return 12 + else: + return 0 + +@mock.patch.object(our_custom_module, "special_char") +def test_function_under_test(self, m_special_char): + """ Test functionality of `function_under_test` + + This mock decorator _temporarily_ overwrittes the :py:func:`special_char` + function within :py:module:`our_custom_module`. We get access to this + mocked object as function argument. Concurrent dependencies of these mock + statements are automatically solved by stestr. + """ + + # Mock the return value of the mocked object to be None + m_special_char.return_value = None + + # Assert that function_under_test returns 0 when special_char returns None. + self.assertEqual(0, function_under_test()) + +``` + +## Running tox tasks + +Running tasks defined in Tox might be a little different than what you are used +to. This is due to the Tango devices running from Docker containers. Typically, +the dependencies are only available inside Docker and not on the host. + +The tasks can thus only be accessed by executing Tox from within a Tango device +Docker container. A simple interactive Docker exec is enough to access them: + +```sh +docker exec -it device-sdp /bin/bash +cd /opt/lofar2.0/tango/devices/ +tox +``` + +For specific tasks the `-e` parameter can be used, in addition, any arguments +can be appended after the tasks. These arguments can be interpreted within the +Tox configuration file as `{posargs}`. Below are a few examples: + +```sh +# Execute unit tests with Python 3.7 and only execute the test_get_version test +# from the TestLofarGit class found within util/test_lofar.py +# Noteworthy, this will also execute test_get_version_tag_dirty due to pattern +# matching. +tox -e py37 util.test_lofar.TestLofarGit.test_get_version_tag +# Execute linting +tox -e pep8 +``` + +## Debugging unit tests + +Debugging works by utilizing the +[virtual environment](https://virtualenv.pypa.io/en/latest/)that Tox creates. +These are placed in the .tox/ directory. Each of these environments carries +the same name as found in _tox.ini_, these match the names used for `-e` +arguments + +Debugging unit tests is done by inserting the following code segment just before +where you think issues occur: + +```python +import pdb; pdb.set_trace() +``` + +Now as said stestr will catch any breakpoints and reraise them so we need to +avoid using stestr while debugging. Simply source the virtual environment +created by tox `source .tox/py37/bin/activate`. You should now see that the +shell $PS1 prompt is modified to indicate the environment is active. + +From here execute `python -m testtools.run` and optionally the specific test +case as command line argument. These test will not run in parallel but support +all other features such as autodiscovery, test fixtures and mocking. + +Any breakpoint will be triggered and you can use the pdb interface +(very similar to gdb) to step through the code, modify and print variables. + +Afterwards simply execute `deactivate` to deactivate the virtual environment. +**DO NOT FORGOT TO REMOVE YOUR `pdb.set_trace()` STATEMENTS AFTERWARDS** + +The best approach to prevent committing `import pdb; pdb.set_trace()` is to +ensure that all unit tests succeed beforehand. \ No newline at end of file diff --git a/devices/tox.ini b/devices/tox.ini index 88d048b1e994ca79ce1727fb3c4d397c45fcbbe7..eb74837103164bf2991a6c768337366d230d09ca 100644 --- a/devices/tox.ini +++ b/devices/tox.ini @@ -17,8 +17,10 @@ deps = -r{toxinidir}/test-requirements.txt -r{toxinidir}/../docker-compose/lofar-device-base/lofar-requirements.txt commands = stestr run {posargs} +; TODO(Corne): Integrate Hacking to customize pep8 rules [testenv:pep8] commands = +; doc8 is only necessary in combination with Sphinx or ReStructuredText (RST) ; doc8 doc/source/ README.rst flake8 ./{posargs} - bandit -r ./ -x test -n5 -ll # -s B320 \ No newline at end of file + bandit -r ./ -x test -n5 -ll # -s B320