Station Client Library

Client library for using Tango Station Control.

Table of Contents:

• Prerequisites
• Installation
• Usage
• Development
• Debug
• Releasenotes

Prerequisites

This package uses PyTango, which wraps the Tango C++ library. You will need to install the Tango C++ headers & library to allow pip to install PyTango as part of this package's requirements:

Debian and Ubuntu provide these natively:

apt-get install libtango-dev

Under Arch, install the tango-cpp AUR package. For other distros and installation methods, see Tango's Linux installation manual.

You will also need the Boost Python C++ headers & libraries to compile PyTango. Under Debian/Ubuntu, obtain these through:

apt-get install libboost-python-dev

Installation

Wheel distributions are available from the gitlab package registry, install using after downloading:

pip install *.whl

Alternatively install latest version on master using:

pip3 install lofar-station-client@git+https://git.astron.nl/lofar2.0/lofar-station-client

Or install directly from the source at any branch or commit:

pip install ./

Usage

For more thorough usage explanation please consult the documentation

Obtain archived monitoring points

To retrieve old values for a monitoring point from the Prometheus database, use:

from lofar_station_client import get_attribute_history
# Metrics from the last hour
get_attribute_history("Pointing_direction_R", "stat/digitalbeam/lba")
# Specify range in epoch
get_attribute_history("Pointing_direction_R", "stat/digitalbeam/lba", start=1655725200.0, end=1655815200.0)

Parse raw UDP packet streams from a station

Suppose you captured statistics or beamlets in a file called packets.raw. You can then parse these and print a brief summary per packet using:

from lofar_station_client.statistics.receiver import FileReceiver

for packet in FileReceiver("packets.raw"):
    print(packet)

You can also process them live from a station, for example:

from lofar_station_client.statistics.receiver import TCPReceiver

for packet in TCPReceiver("cs001c.control.lofar", 5101):
    print(packet)

Development

Be sure to validate your patches will pass the build system by executing tox everytime before you submit.

pip install tox
tox

To ensure your patch will pass documentation and or packaging tests run

tox -e docs
tox -e build

Running integration tests is also possible.

tox -e build
tox -e integration

Debug

Place import pdb; pdb.set_trace() on desired breakpoints and execute:

tox -e debug path.to.test
tox -e debug tests.requests.test_prometheus

Release notes

• 0.18.5 - Compatability with python 3.10 and higher
• 0.18.4 - Compatability with PyTango 9.5.0
• 0.18.3 - Refactoring statistics packets. Moving tango to optional dependency [tango]
• 0.18.2 - Bugfix when closing unused HDF5 files
• 0.18.1 - Flush HDF5 files explicitly, reduce memory usage for XSTs
• 0.18.0 - MultiStationObservation and StationFutures allow multi field observations
• 0.17.3 - Fix hosts and ports to be compatible with consul overlay network
• 0.17.2 - Fix antennafield_device naming after separation in AFL and AFH
• 0.17.1 - Add missing subbands field to statistics data
• 0.17.0 - Add gzip compression to HDF5 files in statistics writer
• 0.16.3 - Fix AntennaField in filename
• 0.16.2 - Fixed MultiStationObservation.all_connected - Renamed SDPPacket.upcast to downcast
• 0.16.1 - Added iterator to Receiver classes - Added antenna_field_index to packet header
• 0.16 - HDF5 attributes: Moved frequency_band and rcu_* from header to matrix - HDF5 attributes: Replaced subband_frequencies by the smaller subband_frequency_range - HDF5 attributes: Removed superfluous nyquist_zones and spectral_inversion attributes - Better error handling when writing HDF5 files and attributes
• 0.15.9 - Fix: read correct attribute for the frequency bands
• 0.15.8 - SST/XST: Account for a signal index offset (HBA1) when collecting statistics
• 0.15.7 - Fix: Recording LBA statistics does not request HBA-only metadata - Fix: Syntax error when querying SDP metadata
• 0.15.6 - Represent BSTs in 488x2 arrays - l2ss-statistics-writer: Fix syntax error in printing help - More resilience against errors when interfacing with Tango
• 0.15.5 - Fix rounding of dataset timestamps, dropped timezone
• 0.15.4 - Fix parsing of BST packets
• 0.15.3 - BSTs: added digital pointing direction, and subband selection
• 0.15.2 - Statistics filenames: added antennafield, moved subband to the end
• 0.15.1 - Infer device names and default port from the specified antenna-field name
• 0.15.0 - Add all_connected method to MultiStationObservation
• 0.14.8 - Optimizing hdf dictionary writing to cause less memory and IO overhead
• 0.14.7 - Refactor LazyDeviceProxy
• 0.14.6 - Removed deprecated StationSSTCollector
• 0.14.5 - Added gn_indices and support for global node indices > 16.
• 0.14.4 - Fixed bug on writer_version retrieval
• 0.14.3 - Added rcu_pcb_id and rcu_pcb_version to Hdf5 file header
• 0.14.2 - Added station_name attribute to Hdf5 file header
• 0.14.1 - Added beamlet.subband_select_RW attribute to BSTHdf5Writer
• 0.14 - Added new attributes to statistics HDF file as well as documentation
• 0.13 - Added lazy connection behavior to devices.LofarDeviceProxy class
• 0.12.
  • Added HDF5Writer class for writing HDF5 statistics file
  • Added Receiver class for creating a file/TCP receiver for retrieving statistics data
  • Added Reader class for parsing and reading statistics file data
  • Added StationSSTCollector for handling Tango device parameters
  • Added Entry file which includes the procedures to launch the statistics writer
• 0.11.2 - MultiStationObservation start() and stop() return the station results
• 0.11.1 - Bug fixes in observation code
• 0.11.
  • Added MultiStationObservation class for a pythonic interface with observations across multiple stations.
  • Added StationObservation class for pythonic interface with observations
  • Added Observation class for pythonic interface with observations
  • Added generic HDF5 file reader
• 0.9.1 - Add last_invalid_packet_exception parameter to StatisticsCollector
• 0.9. - Added devices.LofarDeviceProxy that transparently exposes >2D attributes
• 0.8. - Fixed XST packet parsing.
• 0.7. - Partial rework of DTS outside code, still many possible improvements.
• 0.6. - Correctly transpose XST blocks in XSTCollector.
• 0.5. - Swapped [x][y] for [y][x] dimensionality in get_attribute_history
• 0.4. - Added collectors including XSTCollector, BSTCollector and SSTCollecotr import under from lofar_station_client.statistics import xxx
• 0.x. - Lost to the Void