Skip to content
Snippets Groups Projects

RAP-840 Add Read the Docs

Merged RAP-840 Add Read the Docs
All threads resolved!
Merged Mick Veldhuis requested to merge rap-840-add-docs into main
All threads resolved!
1 file
+ 26
20
Compare changes
  • Side-by-side
  • Inline
+ 26
20
@@ -4,17 +4,17 @@ Overview of the Pipeline
Processing workflow
-------------------
The pre-processing takes place in the ``preprocess`` step, which is a wrapper to DP3. This step consists of four major categories: (1) flagging, (2) demixing, (3) averaging, and (4) demixing. The flagging procedure is comprised of multiple stages of flagging with DP3's PreFlagger and flagging of radiofrequency interference (RFI) with AOFlagger. Subsequently, A-team sources are demixed and the data is averaged in time and frequency. Finally, the visibility data is compressed using the Dysco compression method.
The pre-processing takes place in the ``preprocess`` step, which is a wrapper to DP3. This step consists of four major categories: (1) flagging, (2) demixing, (3) averaging, and (4) demixing. The flagging procedure is comprised of multiple stages of flagging with DP3's PreFlagger and flagging of radio-frequency interference (RFI) with AOFlagger. Subsequently, A-team sources are demixed and the data is averaged in time and frequency. Finally, the visibility data is compressed using the Dysco compression algorithm.
The processing flow is as follows:
The pipeline performs the following operations in order:
* flagging edge channels
* flagging correlation type (e.g., all auto-correlations)
* flagging low-amplitude signals (below 1E-30)
* flagging of RFI
* flagging of interfering radio signals
* demixing of A-team sources
* Averaging in time and frequency
* Dysco compression of visibility data
* averaging in time and frequency
* compression of visibility data
Diagnostics
-----------
@@ -28,31 +28,37 @@ User-defined parameters
**Mandatory parameters:**
* ``msin``: path to the input data (MeasurementSets)
* ``msin``: path to the input data (list of MeasurementSets).
* ``demix_timestep``: number of time steps to average during demixing.
* ``demix_freqstep``: number of channels to average during demixing.
* ``avg_timestep``: this parameter defines the number of time steps that will be averaged.
* ``avg_freqstep``: this parameter defines the number of channels that will be averaged.
**Optional parameters:**
*Flagging parameters:*
* ...
* ``preflag_corrtype``: select a type of correlation to flag, e.g., the auto-correlations (default: ``auto``).
* ``preflag_min_amplitude``: data below this amplitude will be flagged (default: ``1E-30``).
* ``aoflagger_rfistrategy``: the RFI flagging strategy used by AOFlagger (default: ``lofar-default.lua``).
*Options for demixing:*
* ...
*Options for averaging:*
* ``avg_timestep``
* ``avg_freqstep``
* ``demix_skymodel``: the skymodel used by the demixing algorithm (default: ``Ateam.skymodel``).
* ``demix_sources``: the list of sources to demix, e.g., ``[CasA, CygA]``. Note that these sources have to be present in the provided skymodel. (default: ``[]``.)
* ``demix_baselines``: select the baselines used to demix (default: ``[CR]S*&"``).
* ``demix_ignoretarget``: set this parameter to false if you want to project the target away, if set to true the target will be ignored (default: ``false``).
* ``demix_lbfgs_robustdof``: the degrees of freedom of the LBFGS solver noise model (default: ``200``).
* ``demix_lbfgs_historysize``: the history size the LBFGS solver uses to approximate the inverse Hessian (default: ``10``).
*Options for compression:*
* ``dysco_distribution``
* ``dysco_databitrate``
* ``dysco_weightbitrate``
* ``dysco_distribution``: compression distribution used by the Dysco compression algorithm (default: ``TruncatedGaussian``).
* ``dysco_databitrate``: the number of bits per float used to represent the visibility data (default: ``10``).
* ``dysco_weightbitrate``: the number of bits per float used to represent the weights (default: ``12``).
*General pipeine options and performance:*
*Miscellaneous parameters:*
* ``sasid``
* ``dp3_checkparset``
* ``dp3_numthreads``
\ No newline at end of file
* ``sasid``: identifier of the SAS process that called the pipeline; this unique identifier is prefixed with an 'L'.
* ``msin_autoweight``: set this parameter to true when the input consist of raw LOFAR data, this will ensure proper weights are set (default ``true``).
* ``dp3_numthreads``: the number of threads per process used by DP3 (default: ``10``).
Loading