Merge branch 'interface' into 'master'

Add more info regarding output data

See merge request eosc/prefactor3-cwl!64

Former-commit-id: 289e7688

docs/source/target.rst

@@ -14,7 +14,7 @@ This chapter will present the specific steps of the target pipeline in more deta
 All results (diagnostic plots and calibration solutions) will be stored usually in the ``--outdir`` directory specified with your ``cwltool`` or ``toil`` command.
 
     .. image:: targetscheme.png
-    
+
 
 Prepare target, incl. "demixing" (``prep``)
 -------------------------------------------
@@ -72,7 +72,7 @@ The workflow ``gsmcal`` consists of:
     - wide-band statistical flagging (steps ``ms_concat`` and ``aoflag``)
     - checking for bad data chunks (step ``check_unflagged_fraction``)
     - perform the self-calibration against the global skymodel (subworkflow ``calibrate_target``, baseline-dependend smoothing (step ``BLsmooth``) if specified ``do_smooth : true``))
-    
+
 Finalizing the prefactor output (``finalize``)
 ----------------------------------------------
 These steps produce the final data output and many helpful diagnostics.
@@ -89,14 +89,14 @@ The last step also incorporates full `Dysco`_ compression to save disk space. Th
 
 .. note::
     All solutions are written in the h5parm file format via the steps ``H5parm_collector`` and called during all the workflows.
-    
+
     The solutions are stored in the final calibrator solution set ``results/cal_values/cal_solutions.h5``.
 
 Further diagnostics
 -------------------
 
 The ``results`` directory will contain all relevant outputs of the current **prefactor** run, once the pipeline has finished:
-    - fully calibrated data sets
+    - fully calibrated datasets, concatenated with ``num_SBs_per_group`` subbands per MS file and averaged, if desired (see averaging options below). The DATA column of each MS contains the calibrated data (with the direction-independent solutions applied).
     - logfiles in ``results/logs``
     - summary file (JSON format) in ``results/summary``
     - calibration solutions in ``results/cal_values/cal_solutions.h5``
@@ -112,16 +112,16 @@ The following diagnostic help to assess the quality of the data reduction:
     - ``??_uv-coverage_uvdist.png``: fraction of remaining unflagged data versus ``uv``-distance
         .. image:: uv-coverage_uvdist.png
     - ``??_uv_coverage.png``: the ``uv``-coverage of the final data set
-        .. image:: uv-coverage.png    
+        .. image:: uv-coverage.png
     - ``??_structure.png``: plot of the ionospheric `structure function`_ of the processed target field
     .. image:: structure.png
     - ``??-MFS-image.fits``: FITS image of the target field
     .. image:: target_field.png
-    
+
 You can also check the calibration solutions for more details::
 
     $ losoto -i results/cal_values/cal_solutions.h5
-    
+
     Summary of results/cal_values/cal_solutions.h5
 
     Solution set 'calibrator':
@@ -189,11 +189,11 @@ You can also check the calibration solutions for more details::
 For an overall summary it is advised to check the summary logfile::
 
     $ cat results/logs/???_summary.log
-    
+
     *****************************************
     *** prefactor target pipeline summary ***
     *****************************************
-    
+
     Field name: P000+00
 
     User-specified baseline filter: [CR]S*&
@@ -208,21 +208,21 @@ For an overall summary it is advised to check the summary logfile::
 
     Amount of flagged solutions per station and solution table:
     Station   bandpass    clock    faraday  polalign  RMextract TGSSphase
-    CS001HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.00% 
-    CS001HBA1    0.29%     0.00%     0.00%     0.00%     0.00%     0.00% 
-    CS002HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.05% 
-    CS002HBA1    0.29%     0.00%     0.00%     0.00%     0.00%     0.00% 
-    CS003HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.00% 
-    CS003HBA1    0.29%     0.00%     0.00%     0.00%     0.00%     0.05% 
-    CS004HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.05% 
-    CS004HBA1    6.05%     0.00%     0.00%     0.00%     0.00%     0.05% 
-    CS005HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.05% 
-    CS005HBA1    0.39%     0.00%     0.00%     0.00%     0.00%     0.00% 
+    CS001HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.00%
+    CS001HBA1    0.29%     0.00%     0.00%     0.00%     0.00%     0.00%
+    CS002HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.05%
+    CS002HBA1    0.29%     0.00%     0.00%     0.00%     0.00%     0.00%
+    CS003HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.00%
+    CS003HBA1    0.29%     0.00%     0.00%     0.00%     0.00%     0.05%
+    CS004HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.05%
+    CS004HBA1    6.05%     0.00%     0.00%     0.00%     0.00%     0.05%
+    CS005HBA0    0.29%     0.00%     0.00%     0.00%     0.00%     0.05%
+    CS005HBA1    0.39%     0.00%     0.00%     0.00%     0.00%     0.00%
     CS006HBA0    0.29%     0.00%     0.00%     0.00%     0.00%
     CS006HBA1    0.29%     0.00%     0.00%     0.00%     0.00%
 
     Amount of flagged data per station at a given state:
-    Station    initial  prep    Ateam   final 
+    Station    initial  prep    Ateam   final
     CS001HBA0   5.13%   5.41%  11.12%  22.74%
     CS001HBA1   5.13%   5.41%  11.03%  22.51%
     CS002HBA0   5.12%   5.39%  11.39%  23.18%
@@ -235,11 +235,11 @@ For an overall summary it is advised to check the summary logfile::
     CS005HBA1   5.12%  15.36%  20.07%  30.09%
     CS006HBA0 100.00% 100.00% 100.00%
     CS006HBA1 100.00% 100.00% 100.00%
-    
+
     **********
     Summary file is written to: ???_prefactor_target_summary.json
     Summary has been created.
-    
+
 
 User-defined parameter configuration
 ------------------------------------