Skip to content
Snippets Groups Projects
Commit 86c3cc4a authored by Mick Veldhuis's avatar Mick Veldhuis
Browse files

Remove stray docs page

parent 88b75205
No related branches found
No related tags found
1 merge request!13Fix concat_logfiles error
Pipeline #102873 passed
Downloading and Installation
============================
Instructions for downloading and installing the Pre-Processing Pipeline are summarised below and also available on its `GitLab page <https://git.astron.nl/RD/preprocessing-cwl>`_.
Obtaining the Pre-Processing Pipeline
-------------------------------------
The pipeline is written in the `Common Workflow Language (CWL) <https://www.commonwl.org/>`_ and consists of CWL workflows, which can be obtained with:
.. code:: console
$ git clone https://git.astron.nl/RD/preprocessing-cwl
Manual Installation
-------------------
To run the pipeline, you need a CWL runner, such as ``cwltool`` or ``toil``. It's recommended to install these in a virtual environment:
.. code:: console
$ python3 -m venv venv
$ source venv/bin/active
$ pip install cwltool toil[cwl]
Note that it's also necessary to install NodeJS, which can either be installed via your favourite package manager or using the ``nodejs-wheel`` Python package.
Additionally, the Pre-Processing Pipeline depends on the following software for processing:
* `LofarStMan <https://github.com/lofar-astron/LofarStMan>`_
* `DP3 <https://git.astron.nl/RD/DP3>`_
* `AOFlagger <https://gitlab.com/aroffringa/aoflagger>`_
* `SAGECal <https://github.com/nlesc-dirac/sagecal>`_
* `Casacore <https://github.com/casacore/casacore>`_
* `EveryBeam <https://git.astron.nl/RD/EveryBeam>`_
Please follow their respective installation instructions.
Docker Installation
-------------------
To run the pipeline within a Docker container, please build the image located in the ``docker/pipeline`` directory of the repository:
.. code:: console
$ docker build ../.. -f Dockerfile -t preprocess:latest
Please modify and give the image an appropriate tag with the ``-t`` option. This requires one to install Docker on their system, either `Docker Engine <https://docs.docker.com/engine/>`_ (only available for Linux) or `Docker Desktop <https://docs.docker.com/desktop/>`_ (available for Windows, MacOS, and Linux). Instead of Docker, one could also use `Apptainer <https://apptainer.org/>`_ (previously called Singularity) on Linux.
Note: if the dependencies have not been installed manually and the pipeline is not ran inside a container, workflow components will attempt to run inside a Docker or Docker-compatible container environment. In this case, a Docker image is pulled from the repository's container registry.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment