Tango Station Control

Station Control software related to Tango devices.
Please consult the online documentation!

Index

• Installation
  • Prerequisites
  • Bootstrap
• User documentation (ReadTheDocs (Sphinx / ReStructuredText))
• Docker compose & station services documentation
  • Jupyter startup files
  • Tango Prometheus exporter
• Developer Documentation
  • Deployments
  • Versioning
• Source code documentation
  • Attribute wrapper documentation
  • Adding a new tango device
  • HDF5 statistics
• Unit tests
• Integration tests
• Release Notes

Installation

Prerequisites

After checking out this repo, be sure to also check out the submodules:

git submodule init
git submodule update

You will also need:

• docker
• docker-compose
• make
• bash
• dig (dnsutils package on ubuntu/debian)
• wget
• hexdump (bsdextrautils package on ubuntu/debian)

Start dev environment

For local development a dev environment is needed. To setup this environment run

source ./sbin/prepare_dev_env.sh

This will install jumppad, if not present yet as well as creating the docker volume needed to simulate the station nomad cluster.

Afterwards run

jumppad up infra/dev

to start the dev environment including tango.

Nomad is now available at http://localhost:4646/

Bootstrap

The bootstrap procedure is needed only once. First we build all docker containers, and load the initial configuration. This may take a while:

cd docker-compose
make bootstrap

By default bootstrap will configure the station to use simulators. You can lookup alternative configurations in the CDB directory.

Now we can start all containers, and make sure everything is up:

make start
make status

If not, you can inspect why with docker logs <container>. The containers will automatically be restarted on reboot or failure. Stop them explicitly to bring them down (make stop <container>).

Most notably, you will have web interfaces available at:

• http://localhost:8888 (Jupyter Lab)
• http://localhost:3000 (Grafana).

Development

For development, you will need several dependencies including:

git g++ gcc make docker docker-compose shellcheck graphviz python3-dev \
python3-pip python3-tox libboost-python-dev libtango-cpp pkg-config

Please source setup.sh file prior to developing to install any git hooks and setup environment variables.

source setup.sh

Of these docker-compose must be at least 2.0 and Python 3.10 or higher. Alternatively, tox can be installed through pip using pip install tox.

Finally, running unit tests relies on availability of casacore data see: lofar-device-base Dockerfile for details.

Versioning

When changing behavior a new version for Lofar Station Control should be reserved. To do this please follow semantic versioning.

Next change the version in the following places:

1. The VERSION file.
2. Add a Release note for the given version.
3. Once the merge requests is merged to master, add a tag with the version such as v0.3.2 or v0.3.3 etc
4. The tag can be deployed to the environments, manually, through https://git.astron.nl/lofar2.0/tango/-/tags

Release Notes

• 0.24.4 Apply calibration values in reverse order if subband frequencies are decreasing
• 0.24.3 Tune python logging format and parsing
• 0.24.2 Ensure code base is PyTango 9.5 compatible
• 0.24.1 Let all devices emit basic prometheus metrics
• 0.24.0 Allow multiple antenna fields to be used in single observation, This renames the Observation device to ObservationField.
• 0.23.0 Migrate execution environment to nomad
• 0.22.0 Split Antennafield in AFL and AFH devices in order to separate Low-Band and High-Band functionalities Removed Antenna_Type_R attribute from antennafield devices
• 0.21.4 Replace ACC-MIB.mib with SP2-MIB.mib source file in PCON device
• 0.21.3 Added DigitalBeam.Antenna_Usage_Mask_R to expose antennas used in beamforming
• 0.21.2 Removed deprecated "Boot" device (use StationManager now)
• 0.21.1 Implement multi project integration downstream pipeline
• 0.21.0 Use radians instead of degrees when interpreting pointings
• 0.20.5 Manage both polarisations in RCU_band_select_R(W), Antenna_Loss_R, and Frequency_Band_RW
• 0.20.4 Collapse AbstractHierarchyDevice and AbstractHierarchy into one class
• 0.20.3 Fix application of Field_Attenuation_R
• 0.20.2 Support only one parent in hierarchies
• 0.20.1 Create an abstract AntennaMapper class which implements behavior of both AntennaToSdpMapper and AntennaToRecvMapper
• 0.20.0 Complete implementation of station-state transitions in StationManager device. Unified power management under power_hardware_on/off(), dropping prepare_hardware(), disable_hardware(). Replaced device.warm_boot() by device.boot().
• 0.19.0 Ensure requirements.txt are installed when using pip install
• 0.18.3 Many configuration fixes in tango device configs, Fixed APS & EC device port mapping, fixed variable initialization in several devices, Fixed XST device going into fault state, prevent UDP packet loss and verify UDP buffer size for XSTs, Fixed several tests due to use of numpy.array in properties, Implement control hierarchy, Version pin PyASN, Fix code coverage for PyTango devices, Fix beam tracker not starting again after being stopped.
• 0.18.2 Fix documentation links in README
• 0.18.1 Various improvements including: better error handling for commands and resolving a configuration issue related to beamlets
• 0.18.0 Expose attribute related to SDP rings such as FPGA_bf_ring_nof_transport_hops_RW_default and FPGA_ring_use_cable_to_next_rn_RW_default
• 0.17.1 Ensure OPCUA devices reconnect automatically if the connection is lost
• 0.17.0 Add Power Hierarchy state transition
• 0.16.2 Add Power_Parent and Parent_Children properties in LOFAR devices
• 0.16.1 AntennaField: Do not put device in FAULT if an attribute cannot be read/written. AntennaField: Avoid archiving HBA-specific attributes for LBA fields.
• 0.16.0 Observation: Removed antenna mask from specification DigitalBeam: Removed beamlet and antenna selection
• 0.15.0 Split recv device into rcu2h and rcu2l and split recv-sim translator into rcu2h-sim and rcu2l-sim
• 0.14.0 Create async device base and make tilebeam and digitalbeam async device servers, allowing for cooperative multitasking and preventing issues with beamtracking.
• 0.13.1 Upgrade PyTango to 9.4.x and ensure it is installed through requirements.txt
• 0.13.0 Remove all archiver-timescale, hdbppts-cm, hdbppts-es functionalities
• 0.12.1 Add AbstractHierarchy and AbstractHierarchyDevice classes and functionality
• 0.12.0 Add Calibration_SDP_Subband_Weights_<XXX>MHz_R attributes to implement HDF5 calibration tables
• 0.11.2 Fix sleep duration in archiver test
• 0.11.1 Fix event unsubscription in TemperatureManager
• 0.11.0 Added StationManager device
• 0.10.0 Add AntennaToSdpMapper and fpga_sdp_info_* mapped attributes in Antennafield device
• 0.9.0 Statistics writer: moved the whole functionality to lofar-station-client repository
• 0.8.0 Statistics writer: HDF5 format overhaul (removed values, added and moved attributes), Statistics writer: Added --field parameter to record statistics of a specific AntennaField, AntennaField: Added RCU_DTH_on_R, RCU_DTH_freq_R(W), RCU_band_select_R, RCU_attenuator_dB_R.
• 0.7.2 Added sdp.subband_frequency_R, antennafield.Frequency_Band_RW, and support for spectral inversion
• 0.7.1 Add restore backup configuration for Configuration device
• 0.7.0 Raised required Python version to 3.10
• 0.6.0 Changed recv.ANT_mask_RW and recv.ANT_mask_R into 32x3 matrices
• 0.5.1 Add loading and updating methods for Configuration device
• 0.5.0 Add Configuration device
• 0.4.1 Fix for missing SDP attributes for spectral inversion
• 0.4.0 Have most containers report health status and add make await command
• 0.3.1 Fix for applying boot device dsconfig
• 0.3.0 Initial version of deployment scripts and functionality
• 0.2.0 Extend Beamlet device with FPGA source address attributes
• 0.1.2 Fix StatisticsClient accessing last_invalid_packet_exception parameter