Merge branch 'L2SS-259-ci-unit-tests' into 'master'

L2SS-259: Integrate unit tests using Continuous Integration (CI)

Closes L2SS-259

See merge request !58

.gitignore

@@ -2,6 +2,11 @@
 **/.DS_Store
 **/__pycache__
 **/*.pyc
+**/*.egg-info
+**/.tox
+**/.stestr
+**/AUTHORS
+**/ChangeLog
 **/env
 **/vscode-server.tar
 **/.project

.gitlab-ci.yml

+# TODO(Corne): Update this image to use our own registry once building
+#              images is in place.
+image: artefact.skatelescope.org/ska-tango-images/tango-itango:9.3.3.7
+variables:
+  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
+  # The PBR dependency requires a set version, not actually used
+  # Instead `util/lofar_git.py:get_version()` is used.
+  PBR_VERSION: "0.1"
+cache:
+  paths:
+    - .cache/pip
+    - devices/.tox
+stages:
+  - building
+  - linting
+  - unit-tests
+linting:
+  stage: linting
+  script:
+    - cd devices
+    - tox -e pep8
+unit_test:
+  stage: unit-tests
+  before_script:
+    - sudo apt-get update
+    - sudo apt-get install -y git
+  script:
+    - cd devices
+    - tox -e py37

README.md

+# Tango Station Control
+
+Station Control software related to Tango devices.
\ No newline at end of file

devices/.stestr.conf

+[DEFAULT]
+test_path=./test
+top_dir=./

devices/README.md

-# Device wrapper
+# Tango Station Control Device wrappers
 
 This code provides an attribute_wrapper class in place of attributes for tango devices. the attribute wrappers contain additional code 
 that moves a lot of the complexity and redundant code to the background. 

devices/__init__.py

+from util.lofar_git import get_version
+
+__version__ = get_version()
+

devices/requirements.txt

+# the order of packages is of significance, because pip processes them in the
+# order of appearance. Changing the order has an impact on the overall
+# integration process, which may cause wedges in the gate later.
+
+pbr>=2.0 # Apache-2.0

devices/setup.cfg

+[metadata]
+name = TangoStationControl
+summary = LOFAR 2.0 Station Control
+description-file =
+    README.md
+description-content-type = text/x-rst; charset=UTF-8
+author = ASTRON
+home-page = https://astron.nl
+project_urls =
+    Bug Tracker = https://support.astron.nl/jira/projects/L2SS/issues/
+    Source Code = https://git.astron.nl/lofar2.0/tango
+license = Apache-2
+classifier =
+    Environment :: Console
+    License :: Apache Software License
+    Operating System :: POSIX :: Linux
+    Programming Language :: Python
+    Programming Language :: Python :: 3
+    Programming Language :: Python :: 3.6
+    Programming Language :: Python :: 3.7
+    Programming Language :: Python :: 3.8
+    Programming Language :: Python :: 3.9
+
+[files]
+package_dir=./
+
+[entry_points]
+console_scripts =
+    SDP = SDP:main
+    PCC = PCC:main

devices/setup.py

+import setuptools
+
+# Requires: setup.cfg
+setuptools.setup(setup_requires=['pbr>=2.0.0'], pbr=True)

devices/test-requirements.txt

+# the order of packages is of significance, because pip processes them in the
+# order of appearance. Changing the order has an impact on the overall
+# integration process, which may cause wedges in the gate later.
+
+doc8>=0.8.0 # Apache-2.0
+flake8>=3.8.0 # MIT
+bandit>=1.6.0 # Apache-2.0
+hacking>=3.2.0,<3.3.0 # Apache-2.0
+coverage>=5.2.0 # Apache-2.0
+python-subunit>=1.4.0 # Apache-2.0/BSD
+Pygments>=2.6.0
+stestr>=3.0.0 # Apache-2.0
+testscenarios>=0.5.0 # Apache-2.0/BSD
+testtools>=2.4.0 # MIT
+

devices/test/README.md

+# Unit Testing
+
+Is the procedure of testing individual units of code for fit for use. Often
+this entails isolating specific segments and testing the behavior given a
+certain range of different inputs. Arguably, the most effective forms of unit
+testing  deliberately uses _edge cases_ and makes assumptions about any other
+sections of code that are not currently under test.
+
+First the underlying technologies as well as how they are applied in Tango
+Station Control are discussed. If this seems uninteresting, or you are already
+familiar with these concepts you can skip to
+[Running tox tasks](#running tox tasks) and
+[Debugging unit tests](#debugging-unit-tests).
+
+### Table of Contents:
+
+- [Tox](#tox)
+- [Testing approach](#testing-approach)
+- [Mocking](#mocking)
+- [Running tox tasks](#running-tox-tasks)
+- [Debugging unit tests](#debugging-unit-tests)
+
+## Tox
+
+[Tox](https://tox.readthedocs.io/en/latest/) is a commandline tool to simplify
+running Python tasks such as linting and unit testing. Using a simple
+[configuration file](../tox.ini) it can setup a
+[virtual environment](https://virtualenv.pypa.io/en/latest/) that automatically
+installs dependencies before executing the task. These environments persist
+after the task finishes preventing the excessive downloading and installation
+of dependencies.
+
+The Tox environments in this project are configured to install any dependencies
+listed in [test-requirements.txt](../test-requirements.txt) and
+[lofar-requirements.txt](../../docker-compose/lofar-device-base/lofar-requirements.txt)
+this can also  easily be verified within our [configuration file](../tox.ini).
+
+## Testing approach
+
+For testing [stestr](https://stestr.readthedocs.io/en/latest/) is used this
+tool  has main advantage the utilization of a highly parallelized unit test
+runner that automatically scales to the number of simultaneous threads
+supported by the host processor. Other features include automatic test
+discovery and pattern matching to only execute a subset of tests.
+
+However, stestr is incompatible with using breakpoints
+(through [pdb](https://docs.python.org/3/library/pdb.html)) directly and  will
+simply fail the test upon encountering one. The
+[debugging unit tests](#debugging-unit-tests) section describes how to mitigate
+this and still debug individual unit tests effectively.
+
+All tests can be found in this test folder and all tests must inherit from
+`TestCase`, found in [base.py](base.py). This ensures that any test fixtures run
+before and after tests. These test fixtures allow to restore any global modified
+state, such as those of static variables. It is the task of the developer
+writing the unit test to ensure that any global modification is restored.
+
+When writing unit tests it is best practice to mimic the directory structure of
+the original project inside the test directory. Similarly, copying the filenames
+and adding _test__ to the beginning. Below is an example:
+
+* root
+  * database
+    * database_manager.py
+  * test
+    * base.py
+    * database
+        * test_database_manager.py
+
+## Mocking
+
+Contrary to many other programming languages, it is entirely possible to
+modify **any** function, object, file, import at runtime. This allows for a
+great deal of flexibility but also simplicity in the case of unit tests.
+
+Isolating functions is as simple as mocking any of the classes or functions it
+uses and modifying its return values such as specific behavior can be tested.
+Below is a simple demonstration mocking the return value of a function using the
+mock decorator. For more detailed explanations see
+[the official documentation](https://docs.python.org/3/library/unittest.mock.html).
+
+```python
+from unittest import mock
+
+# We pretend that `our_custom_module` contains a function `special_char`
+import our_custom_module
+
+def function_under_test(self):
+    if our_custom_module.special_char():
+        return 12
+    else:
+        return 0
+
+@mock.patch.object(our_custom_module, "special_char")
+def test_function_under_test(self, m_special_char):
+    """ Test functionality of `function_under_test`
+
+    This mock decorator _temporarily_ overwrites the :py:func:`special_char`
+    function within :py:module:`our_custom_module`. We get access to this
+    mocked object as function argument. Concurrent dependencies of these mock
+    statements are automatically solved by stestr.
+    """
+
+    # Mock the return value of the mocked object to be None
+    m_special_char.return_value = None
+
+    # Assert that function_under_test returns 0 when special_char returns None.
+    self.assertEqual(0, function_under_test())
+
+```
+
+## Running tox tasks
+
+Running tasks defined in Tox might be a little different than what you are used
+to. This is due to the Tango devices running from Docker containers. Typically,
+the dependencies are only available inside Docker and not on the host.
+
+The tasks can thus only be accessed by executing Tox from within a Tango device
+Docker container. A simple interactive Docker exec is enough to access them:
+
+```sh
+docker exec -it device-sdp /bin/bash
+cd /opt/lofar2.0/tango/devices/
+tox
+```
+
+For specific tasks the `-e` parameter can be used, in addition, any arguments
+can be appended after the tasks. These arguments can be interpreted within the
+Tox  configuration file as `{posargs}`. Below are a few examples:
+
+```sh
+# Execute unit tests with Python 3.7 and only execute the test_get_version test
+# from the TestLofarGit class found within util/test_lofar.py
+# Noteworthy, this will also execute test_get_version_tag_dirty due to pattern
+# matching.
+tox -e py37 util.test_lofar.TestLofarGit.test_get_version_tag
+# Execute linting
+tox -e pep8
+```
+
+## Debugging unit tests
+
+Debugging works by utilizing the
+[virtual environment](https://virtualenv.pypa.io/en/latest/)that Tox creates.
+These are placed in the .tox/ directory. Each of these environments carries
+the same name as found in _tox.ini_, these match the names used for `-e`
+arguments
+
+Debugging unit tests is done by inserting the following code segment just before
+where you think issues occur:
+
+```python
+import pdb; pdb.set_trace()
+```
+
+Now as said stestr will catch any breakpoints and reraise them so we need to
+avoid using stestr while debugging. Simply source the virtual environment
+created by tox `source .tox/py37/bin/activate`. You should now see that the
+shell $PS1 prompt is modified to indicate the environment is active.
+
+From here execute `python -m testtools.run` and optionally the specific test
+case as command line argument. These test will not run in parallel but support
+all other features such as autodiscovery, test fixtures and mocking.
+
+Any breakpoint will be triggered and you can use the pdb interface
+(very similar to gdb) to step through the code, modify and print variables.
+
+Afterwards simply execute `deactivate` to deactivate the virtual environment.
+**DO NOT FORGOT TO REMOVE YOUR `pdb.set_trace()` STATEMENTS AFTERWARDS**
+
+The best approach to prevent committing `import pdb; pdb.set_trace()` is to
+ensure that all unit tests succeed beforehand.

devices/test/base.py

+# -*- coding: utf-8 -*-
+#
+# This file is part of the LOFAR 2.0 Station Software
+#
+#
+#
+# Distributed under the terms of the APACHE license.
+# See LICENSE.txt for more info.
+
+import unittest
+import testscenarios
+
+
+class BaseTestCase(testscenarios.WithScenarios, unittest.TestCase):
+    """Test base class."""
+
+    def setUp(self):
+        super().setUp()
+
+
+class TestCase(BaseTestCase):
+    """Test case base class for all unit tests."""
+
+    def setUp(self):
+        super().setUp()

devices/test/util/test_lofar_git.py

+# -*- coding: utf-8 -*-
+#
+# This file is part of the LOFAR 2.0 Station Software
+#
+#
+#
+# Distributed under the terms of the APACHE license.
+# See LICENSE.txt for more info.
+
+import git
+from unittest import mock
+
+from util import lofar_git
+
+from test import base
+
+
+class TestLofarGit(base.TestCase):
+
+    def setUp(self):
+        super(TestLofarGit, self).setUp()
+
+        # Clear the cache as this function of lofar_git uses LRU decorator
+        # This is a good demonstration of how unit tests in Python can have
+        # permanent effects, typically fixtures are needed to restore these.
+        lofar_git.get_version.cache_clear()
+
+    def test_get_version(self):
+        """Test if attributes of get_repo are correctly used by get_version"""
+
+        with mock.patch.object(lofar_git, 'get_repo') as m_get_repo:
+            m_commit = mock.Mock()
+            m_commit.return_value = "123456"
+
+            m_is_dirty = mock.Mock()
+            m_is_dirty.return_value = True
+
+            m_get_repo.return_value = mock.Mock(
+                active_branch="main", commit=m_commit, tags=[],
+                is_dirty=m_is_dirty)
+
+            # No need for special string equal in Python
+            self.assertEqual("*main [123456]", lofar_git.get_version())
+
+    def test_get_version_tag(self):
+        """Test if get_version determines production_ready for tagged commit"""
+
+        with mock.patch.object(lofar_git, 'get_repo') as m_get_repo:
+            m_commit = mock.Mock()
+            m_commit.return_value = "123456"
+
+            m_is_dirty = mock.Mock()
+            m_is_dirty.return_value = False
+
+            m_tag = mock.Mock(commit="123456")
+            m_tag.__str__ = mock.Mock(return_value= "version-1.2")
+
+            m_get_repo.return_value = mock.Mock(
+                active_branch="main", commit=m_commit,
+                tags=[m_tag], is_dirty=m_is_dirty)
+
+            self.assertEqual("version-1.2", lofar_git.get_version())
+
+    @mock.patch.object(lofar_git, 'get_repo')
+    def test_get_version_tag_dirty(self, m_get_repo):
+        """Test if get_version determines dirty tagged commit"""
+
+        m_commit = mock.Mock()
+        m_commit.return_value = "123456"
+
+        m_is_dirty = mock.Mock()
+        m_is_dirty.return_value = False
+
+        m_tag = mock.Mock(commit="123456")
+        m_tag.__str__ = mock.Mock(return_value= "version-1.2")
+
+        # Now m_get_repo is mocked using a decorator
+        m_get_repo.return_value = mock.Mock(
+            active_branch="main", commit=m_commit,
+            tags=[m_tag], is_dirty=m_is_dirty)
+
+        self.assertEqual("version-1.2", lofar_git.get_version())
+
+    def test_catch_repo_error(self):
+        """Test if invalid git directories will raise error"""
+
+        with mock.patch.object(lofar_git, 'get_repo') as m_get_repo:
+
+            # Configure lofar_git.get_repo to raise InvalidGitRepositoryError
+            m_get_repo.side_effect = git.InvalidGitRepositoryError
+
+            # Test that error is raised by get_version
+            self.assertRaises(
+                git.InvalidGitRepositoryError, lofar_git.get_version)

devices/tox.ini

+[tox]
+minversion = 2.0
+envlist = py36,py37,py38,py39,pep8
+skipsdist = True
+
+[testenv]
+usedevelop = True
+sitepackages = True
+install_command = pip3 install {opts} {packages}
+setenv =
+   VIRTUAL_ENV={envdir}
+   PYTHONWARNINGS=default::DeprecationWarning
+   OS_STDOUT_CAPTURE=1
+   OS_STDERR_CAPTURE=1
+   OS_TEST_TIMEOUT=60
+deps = -r{toxinidir}/test-requirements.txt
+    -r{toxinidir}/../docker-compose/lofar-device-base/lofar-requirements.txt
+commands = stestr run {posargs}
+
+; TODO(Corne): Integrate Hacking to customize pep8 rules
+[testenv:pep8]
+commands =
+; doc8 is only necessary in combination with Sphinx or ReStructuredText (RST)
+;    doc8 doc/source/ README.rst
+    flake8
+    bandit -r devices/ clients/ common/ examples/ util/ -n5 -ll
+
+[flake8]
+filename = *.py,.stestr.conf,.txt
+select = W292
+exclude=.tox,.egg-info