TMSS

The TMSS project is developing a new software application for the specification, scheduling, and administration of LOFAR observations.

See the TMSS Software Design for more details.

Index

• Getting Started
  • Prerequisites
  • Installing and running TMSS
• Running TMSS with MAC Scheduler
• Continuous Integration
  • Unit Test
  • Integration Test
• Continuous Delivery
  • Test Deployment
  • Production Deployment

Getting Started

To make changes in the Lofar Repo you need a JIRA-ticket, which are for TMSS typical identified as TMSS-[ID]. For example TMSS-179. Get the code to your local Linux machine

git clone https://git.astron.nl/ro/lofar.git
cd lofar
git checkout -b TMSS-179 master

Now you can make changes in your local TMSS-[ID] branch. When committing changed don't forget to add TMSS-[ID] in your comment

git commit -m "TMSS-179: A pretty nice feature"

Prerequisites

Before you can build the TMSS component, you need to install some packages, like The Django and Postgress. Also you need to be sure that some environment variables are set correctly.
The next example shows install of packages in CentOs. You can do the same for Ubuntu with apt-install, some package names are different

Old Centos 7.6 dependencies

This might no longer work, the ci images have been updated to Ubuntu 22.04

yum install -y epel-release
yum install -y cmake gcc-c++ make log4cplus log4cplus-devel python3 python3-libs python3-devel boost readline-devel boost-devel binutils-devel boost-python36 boost-python36-devel gettext which openldap-devel npm nodejs git java-11-openjdk
yum erase -y postgresql postgresql-server postgresql-devel
yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
yum install -y postgresql96 postgresql96-server postgresql96-devel

For install postgresql-server > 9.2 on Centos7 you need to check https://www.postgresql.org/download/linux/redhat/ how to to it.

Ubuntu 22.04 dependencies

Install the packages as listed in docker/lofar-ci/Dockerfile_ci_base_ubuntu and docker/lofar-ci/Dockerfile_ci_tmss

Install pip modules as listed in docker/lofar-ci/Dockerfile_ci_tmss

Setting the path

cd $/HOME/lofar
mkdir -p build_tmss/gnucxx11_opt
cd $HOME/lofar/build/gnucxx_opt
export LD_LIBRARY_PATH=/opt/casacore/lib:`pwd`/lib:$LD_LIBRARY_PATH
export PATH=/opt/casacore/bin:`pwd`/bin:`pwd`/sbin:$PATH
export PYTHONPATH=/opt/casacore/lib:`pwd`/lib/python3.6/site-packages:$PYTHONPATH

Installing and running TMSS

Build and install TMSS

cd $/HOME/lofar/build_tmss/gnucxx11_opt
cmake -DBUILD_PACKAGES=TMSS ../..
make -j $(nproc)
make install

Running TMSS

Now run a TMSS instance on your local machine:

cd installed 
# That's $HOME/lofar/build/gnucxx11_opt/installed
source ./lofarinit.sh
tmss_test_environment -p 8000

Ok, TMSS should be running, check it at: http://127.0.0.1:8000/api

Running TMSS with MAC Scheduler

One of the clients of TMSS is the MAC Scheduler. The MAC Scheduler will poll TMSS for new observations.

The MAC Scheduler will be build in a docker container. The next steps gives the "MAC build" instructions on your local machine cd to you local source root (that's $HOME/lofar in examples used on this page)

docker build -t ci_base -f Docker/lofar-ci/Dockerfile_ci_base .
docker build -t ci_mac -f Docker/lofar-ci/Dockerfile_ci_mac .
docker run --rm -it --net=host -v $HOME/lofar:$HOME/lofar ci_mac

Now you're running the ci_mac image with your local sources mounted into it. That way you can use your favorite editor on your local machine, and commit changes to git from your local machine using your own credentials etc.Ok, let's build MACScheduler:

Run in the ci_mac docker container, make sure you use another build folder:

cd $HOME/lofar
mkdir -p build_mac/gnucxx11_opt
cd build_mac/gnucxx11_opt
cmake -DBUILD_PACKAGES=MainCU -DUSE_LOG4CPLUS=false -DWINCC_ROOT_DIR=/opt/WinCC_OA/3.14/ -DBLITZ_ROOT_DIR=/opt/blitz/ ../..
make -j 5
make install

Now it's build (in the container) Let's run it (in the container)

cd installed
source ./lofarinit.sh
MACScheduler

Continuous Integration

Every commit/push of code to the remote repo branch in gitlab will result in starting the Continuous Integration gitlab 'build' pipeline which is described in .gitlab-ci.yml This pipeline consist of 6 main stages:

1. Prepare
   • LTA docker image
   • SAS docker image
2. Build
   • LTA ingest
   • RA Services
   • TMSS
3. Unit test
   • LTA ingest
   • RA Services
   • TMSS
4. Integration Test
   • LTA ingest
   • RA Services
   • TMSS
5. Create Docker Container
   • TMSS and push to Nexus Repository
6. Deploy (manual action)
   • TMSS Test
   • TMSS Acceptance

Unit Test

Unit Test can be run with the command ctest that's how the 'build' pipeline will execute it. For development convenience, it is also possible to run the TMSS python unittest in PyCharm IDE.

See the TMSS landscape overview.

Integration Test

Integration tests are python unit test with @integration_test decorator. The session authentication test is a typical integration test for TMSS.

Continuous Delivery

The Deployment can only be executed manually by pushing the button after the gitlab pipeline has executed most relevant stages of the pipeline successfully.

Test Deployment

Currently, TMSS has no test or acceptance environments, any references to them should be ignored as they do not exist anymore (2023-04-21).