- identify known issues and apply them if necessary. It uses in place updates to prevent copying the measurement set
Part of this pipeline identifies known issues and applies corrections if possible. It uses in place updates to prevent copying the measurement set.
- in place update does not work well with CWL conditional steps, so a small bash script is used which either calls the fixing script or does nothing depending if the specific issue for the step was found
- Compress the measurement set with Dysco
> **_NOTE:_** In-place update does not work well with CWL conditional steps, so a small bash script is used which either calls the fixing script or does nothing depending if the specific issue for the step was found
- Produces inspection plots and collects metrics to quantify the amount of missing/flagged data
The checks that get performed are:
- abort if the target is the Sun
- run aoflagger if the input is raw data
- apply corrections based on the time of observation. These can be:
- FIX_ANTENNA_TABLE
- FIX_WEIGHT_SPECTRUM
- FIX_BROKEN_TILES
- FIX_STATION_ADDER
There are also some time ranges that cannot be solved. These issues include:
- INACCURATE_FLAGGING_LBA
- FAULTY_LBA_CALIBRATION_TABLES
- STATION_SENSITIVITY_ISSUE
- DELAY_COMPENSATION_ISSUE
The measurement set is then compressed the with Dysco if it was not already compressed. After all measure ment sets have been compressed, inspection plots are produced and metrics to quantify the amount of missing/flagged data are collected.
> **_NOTE:_** Some plots may be skipped if the required metadata to make the plot is missing. This is often the case for "unspecified" datasets.
When a plot is skipped, this is printed to stderr as a warning and thus is shown in the ATDB log file.
