ATDB-LDV
Apertif Task Database for LOFAR Data Valorization
Test Environment on sdc.astron.nl:
main GUI:
admin interface:
REST API
- workflows: http://localhost:8000/atdb/workflows/
- tasks: http://localhost:8000/atdb/tasks/
Micro Services (in separate repo)
Confluence
Confluence Page:
Workflow: This diagram roughly serves as the specs for adapting ATDB for LDV.
Overview Diagrams (under construction)
These diagrams are the core of the ATDB documentation. Currently they are still mostly the original ATDB diagrams. They will be adapted and kept in sync once the ATDB-LDV structure solifies a bit more.
Datamodel:
Workflow/Status Diagram:
Deployment Diagram:
CI/CD (semi) automatic deploy in Docker
For the `master' branch there is a CI/CD pipeline in place which builds and deploys the backend at https://sdc.astron.nl:5554/atdb/
The deploy step requires pushing the 'play' button in the gitlab pipelines section. This is done to not have a mandatory deploy for every minor commmit.
Manual deploy in Docker (alternative to CI/CD)
initial
> cd ~/my_docker
> cd atdb-ldv
> git clone https://git.astron.nl/astron-sdc/atdb-ldv.gitupdate:
> export DOCKER_BUILD_DIR=$HOME/my_docker/atdb-ldv/atdb-ldv/atdb
> export DOCKER_COMPOSE_DIR=$DOCKER_BUILD_DIR/docker
> cd $DOCKER_BUILD_DIR
> git pull
> docker build -t atdb-ldv:latest .
> cd $DOCKER_COMPOSE_DIR
> docker-compose -p atdb up -dDatabase migrations
When the models.py is changed, then the database must be migrated.
This is how to do that manually for a persistent database in a Docker container.
> docker exec -it atdb-ldv python manage.py makemigrations --settings atdb.settings.docker_sdc
> docker exec -it atdb-ldv python manage.py migrate --settings atdb.settings.docker_sdc Postgres Database operations
dump a copy of the database
> cd $HOME/shared
> docker exec -it atdb-ldv-postgres pg_dump -U postgres -O atdbldv > atdb_ldv.sqlload a database (warning, this overwrites the database)
> docker exec -it atdb-ldv-postgres psql --command "drop database atdbldv"
> docker exec -it atdb-ldv-postgres psql --command "create database atdbldv"
> docker exec -it atdb-ldv-postgres psql -U postgres atdbldv -f /shared/atdb_ldv.sql
> docker exec atdb-ldv-postgres psql --command "grant ALL ON DATABASE atdbldv TO dbadmin;" Original ATDB diagrams
These diagrams can be used as inspiration and guidance to the original ATDB services, which can be harvested for functionality.
