Instructions for downloading and installing the Pre-Processing Pipeline are summarised below and also available on its `GitLab page <https://git.astron.nl/RD/preprocessing-cwl>`_.
Obtaining the Pre-Processing Pipeline
-------------------------------------
The pipeline is written in the `Common Workflow Language (CWL) <https://www.commonwl.org/>`_ and consists of CWL workflows, which can be obtained with:
To run the pipeline, you need a CWL runner, such as ``cwltool`` or ``toil``. It's recommended to install these in a virtual environment:
.. code:: console
$ python3 -m venv venv
$ source venv/bin/active
$ pip install cwltool toil[cwl]
Note that it's also necessary to install NodeJS, which can either be installed via your favourite package manager or using the ``nodejs-wheel`` Python package.
Additionally, the Pre-Processing Pipeline depends on the following software for processing:
Please modify and give the image an appropriate tag with the ``-t`` option. This requires one to install Docker on their system, either `Docker Engine <https://docs.docker.com/engine/>`_ (only available for Linux) or `Docker Desktop <https://docs.docker.com/desktop/>`_ (available for Windows, MacOS, and Linux). Instead of Docker, one could also use `Apptainer <https://apptainer.org/>`_ (previously called Singularity) on Linux.
Note: if the dependencies have not been installed manually and the pipeline is not ran inside a container, workflow components will attempt to run inside a Docker or Docker-compatible container environment. In this case, a Docker image is pulled from the repository's container registry.