Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • L2SS-1914-fix_job_dispatch
  • TMSS-3170
  • TMSS-3167
  • TMSS-3161
  • TMSS-3158-Front-End-Only-Allow-Changing-Again
  • TMSS-3133
  • TMSS-3319-Fix-Templates
  • test-fix-deploy
  • TMSS-3134
  • TMSS-2872
  • defer-state
  • add-custom-monitoring-points
  • TMSS-3101-Front-End-Only
  • TMSS-984-choices
  • SDC-1400-Front-End-Only
  • TMSS-3079-PII
  • TMSS-2936
  • check-for-max-244-subbands
  • TMSS-2927---Front-End-Only-PXII
  • Before-Remove-TMSS
  • LOFAR-Release-4_4_318 protected
  • LOFAR-Release-4_4_317 protected
  • LOFAR-Release-4_4_316 protected
  • LOFAR-Release-4_4_315 protected
  • LOFAR-Release-4_4_314 protected
  • LOFAR-Release-4_4_313 protected
  • LOFAR-Release-4_4_312 protected
  • LOFAR-Release-4_4_311 protected
  • LOFAR-Release-4_4_310 protected
  • LOFAR-Release-4_4_309 protected
  • LOFAR-Release-4_4_308 protected
  • LOFAR-Release-4_4_307 protected
  • LOFAR-Release-4_4_306 protected
  • LOFAR-Release-4_4_304 protected
  • LOFAR-Release-4_4_303 protected
  • LOFAR-Release-4_4_302 protected
  • LOFAR-Release-4_4_301 protected
  • LOFAR-Release-4_4_300 protected
  • LOFAR-Release-4_4_299 protected
40 results

TMSS

  • Clone with SSH
  • Clone with HTTPS
  • Jorrit Schaap's avatar
    TMSS-725: added a SubtaskAllowedStateTransitions table, populate it with the...
    Jorrit Schaap authored
    TMSS-725: added a SubtaskAllowedStateTransitions table, populate it with the allowed state transitions, and raise an exception if a user tries to submit an illegal state transition
    cc80250f
    History
    Code owners
    Assign users and groups as approvers for specific file changes. Learn more.

    TMSS

    The TMSS project is developing a new software application for the specification, scheduling, and administration of LOFAR observations.

    See the TMSS Software Design for more details.

    Getting Started

    To make changes in the Lofar Repo you need a JIRA-ticket, which are for TMSS typical identified as TMSS-[ID]. For example TMSS-179. Get the code to your local Linux machine

    git clone https://git.astron.nl/ro/lofar.git
    cd lofar
    git checkout -b TMSS-179 master

    Now you can make changes in your local TMSS-[ID] branch. When committing changed don't forget to add TMSS-[ID] in your comment

    git commit -m "TMSS-179: A pretty nice feature"

    Prerequisites

    Before you can build the TMSS component, you need to install some packages, like The Django and Postgress. Also you need to be sure that some environment variables are set correctly.
    The next example shows install of packages in CentOs. You can do the same for Ubuntu with apt-install, some package names are different

    yum install -y epel-release
    yum install -y cmake gcc-c++ make log4cplus log4cplus-devel python3 python3-libs python3-devel boost readline-devel boost-devel binutils-devel boost-python36 boost-python36-devel gettext which openldap-devel npm nodejs git java-11-openjdk
    yum erase -y postgresql postgresql-server postgresql-devel
    yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
    yum install -y postgresql96 postgresql96-server postgresql96-devel
    RUN pip3 install cython kombu lxml requests pygcn xmljson mysql-connector-python python-dateutil django djangorestframework djangorestframework-xml ldap==1.0.2 flask fabric coverage python-qpid-proton PyGreSQL numpy h5py psycopg2 testing.postgresql Flask-Testing scipy Markdown django-filter python-ldap python-ldap-test ldap3 djangorestframework django-jsonforms django-json-widget django-jsoneditor drf-yasg flex swagger-spec-validator django-auth-ldap mozilla-django-oidc jsonschema

    For install postgresql-server > 9.2 on Centos7 you need to check https://www.postgresql.org/download/linux/redhat/ how to to it.

    Setting the path

    cd $/HOME/lofar
    mkdir -p build_tmss/gnucxx11_opt
    cd $HOME/lofar/build/gnucxx_opt
    export LD_LIBRARY_PATH=/opt/casacore/lib:`pwd`/lib:$LD_LIBRARY_PATH
    export PATH=/opt/casacore/bin:`pwd`/bin:`pwd`/sbin:$PATH
    export PYTHONPATH=/opt/casacore/lib:`pwd`/lib/python3.6/site-packages:$PYTHONPATH

    Installing and running TMSS

    Build and install TMSS

    cd $/HOME/lofar/build_tmss/gnucxx11_opt
    cmake -DBUILD_PACKAGES=TMSS ../..
    make -j 5
    make install

    Running TMSS

    Now run a TMSS instance on your local machine:

    cd installed 
    # That's $HOME/lofar/build/gnucxx11_opt/installed
    source ./lofarinit.sh
    tmss_test_environment -p 8000

    Ok, TMSS should be running, check it at: http://127.0.0.1:8000/api

    Running TMSS with MAC Scheduler

    One of the clients of TMSS is the MAC Scheduler. The MAC Scheduler will poll TMSS for new observations.

    The MAC Scheduler will be build in a docker container. The next steps gives the "MAC build" instructions on your local machine cd to you local source root (that's $HOME/lofar in examples used on this page)

    docker build -t ci_base -f Docker/lofar-ci/Dockerfile_ci_base .
    docker build -t ci_mac -f Docker/lofar-ci/Dockerfile_ci_mac .
    docker run --rm -it --net=host -v $HOME/lofar:$HOME/lofar ci_mac

    Now you're running the ci_mac image with your local sources mounted into it. That way you can use your favorite editor on your local machine, and commit changes to git from your local machine using your own credentials etc.Ok, let's build MACScheduler:

    Run in the ci_mac docker container, make sure you use another build folder:

    cd $HOME/lofar
    mkdir -p build_mac/gnucxx11_opt
    cd build_mac/gnucxx11_opt
    cmake -DBUILD_PACKAGES=MainCU -DUSE_LOG4CPLUS=false -DWINCC_ROOT_DIR=/opt/WinCC_OA/3.14/ -DBLITZ_ROOT_DIR=/opt/blitz/ ../..
    make -j 5
    make install

    Now it's build (in the container) Let's run it (in the container)

    cd installed
    source ./lofarinit.sh
    MACScheduler

    Continuous Integration

    Every commit/push of code to the remote repo branch in gitlab will result in starting the Continuous Integration gitlab 'build' pipeline which is described in .gitlab-ci.yml This pipeline consist of 6 main stages:

    1. Prepare
      • LTA docker image
      • SAS docker image
    2. Build
      • LTA ingest
      • RA Services
      • TMSS
    3. Unit test
      • LTA ingest
      • RA Services
      • TMSS
    4. Integration Test
      • LTA ingest
      • RA Services
      • TMSS
    5. Create Docker Container
      • TMSS and push to Nexus Repository
    6. Deploy (manual action)
      • TMSS Test
      • TMSS Acceptance

    Unit Test

    Unit Test can be run with the command ctest that's how the 'build' pipeline will execute it. For development convenience, it is also possible to run the TMSS python unittest in PyCharm IDE.

    See the TMSS landscape overview.

    Integration Test

    Integration tests are python unit test with @integration_test decorator. The session authentication test is a typical integration test for TMSS.

    Deployment

    The Deployment can only be executed manually by pushing the button after the gitlab pipeline has accomplished all stages of the pipeline successfully. Deployment of TMSS consist of pushing the TMSS docker container to Lofar Test environment scu199.control.lofar and to the Acceptance environment tmss-ua.control.lofar.