Merge branch 'dev-nico' into 'master'

Dev nico

See merge request !6

.gitlab-ci.yml

@@ -47,7 +47,7 @@ docker-deploy-master:
     - chmod 644 ~/.ssh/known_hosts
   script:
     - ssh -o StrictHostKeyChecking=no vermaas@sdc.astron.nl "docker pull "$CI_REGISTRY_IMAGE""
-    - ssh -o StrictHostKeyChecking=no vermaas@sdc.astron.nl "docker-compose -p atdb -f /docker_compose/atdb-ldv/docker-compose-cd.yml up -d"
+    - ssh -o StrictHostKeyChecking=no vermaas@sdc.astron.nl "docker-compose -p atdb -f /docker_compose/atdb-ldv/docker-compose-dev-cd.yml up -d"
   when: manual
   only:
     - master

README.md

@@ -17,7 +17,8 @@ These diagrams roughly serves as the specifications for adapting ATDB for LDV.
   * workflow: https://support.astron.nl/confluence/display/LDV/WORKFLOW
   * datamodel: https://dbdiagram.io/d/5ffc5fb180d742080a35d560
 
-## Overview Diagrams
+## Overview Diagrams (current implementation)
+These diagrams show the current implementation and are kept up-to-date.
 
 ### Datamodel:
   * https://drive.google.com/file/d/1v5hMBQS0jT8DQJwySVISfRa1zF4o0fCQ/view?usp=sharing  
@@ -29,7 +30,7 @@ These diagrams roughly serves as the specifications for adapting ATDB for LDV.
 
 ![](atdb/docs/ATDB-LDV%20Workflow%20Diagram.png)
   
-### Test Environment on sdc-dev.astron.nl:
+## Test Environment on sdc-dev.astron.nl:
 
 ### main GUI:
   * https://sdc.astron.nl:5554/atdb/
@@ -43,29 +44,32 @@ serializers:
   * tasks: http://sdc.astron.nl:5554/atdb/tasks/
 
 get_size:
+Return the sum the sizes of all tasks with a given list of statusses 
   * https://sdc.astron.nl:5554/atdb/tasks/get_size
   * https://sdc.astron.nl:5554/atdb/tasks/get_size?status__in=staged,processing,processed,validating,validated,ingesting,removing,removed
 
+## Build & Deploy
+
 ### Deployment Diagram:
   * https://drive.google.com/file/d/1_j9Fp505pZTxcmzAEdgftdPkoIFrKfAX/view?usp=sharing
   
 ![](atdb/docs/ATDB-LDV%20-%20Deployment%20Diagram.png)
   
-## CI/CD (semi) automatic deploy in Docker
+### CI/CD (semi) automatic deploy in Docker
 For the `master` branch there is a CI/CD pipeline in place which builds and deploys the backend at https://sdc.astron.nl:5554/atdb/ 
 
 The deploy step requires pushing the 'play' button in the gitlab pipelines section. 
 This is done to not have a mandatory deploy for every minor commmit.
 
-## Manual deploy in Docker (alternative to CI/CD)
+### Manual deploy in Docker (alternative to CI/CD)
 
-### initial
+#### initial
 
     > cd ~/my_docker
     > cd atdb-ldv
     > git clone https://git.astron.nl/astron-sdc/atdb-ldv.git
 
-### update:
+#### update:
 
     > export DOCKER_BUILD_DIR=$HOME/my_docker/atdb-ldv/atdb-ldv/atdb
     > export DOCKER_COMPOSE_DIR=$DOCKER_BUILD_DIR/docker
@@ -75,7 +79,7 @@ This is done to not have a mandatory deploy for every minor commmit.
     > cd $DOCKER_COMPOSE_DIR
     > docker-compose -p atdb up -d
     
-### Database changes and migrations
+#### Database changes and migrations
 When the `models.py` is changed, then the database must be migrated.
 This is the procedure for that.
 

atdb/docker-compose-cd.yml → atdb/docker/docker-compose-dev-cd.yml

@@ -34,7 +34,7 @@ services:
       - "traefik.enable=true"
       - "traefik.http.routers.atdb-backend.entryPoints=atdb-ldv"
       - "traefik.http.routers.atdb-backend.service=atdb-backend"
-      - "traefik.http.routers.atdb-backend.rule=Host(`sdc.astron.nl`) && PathPrefix(`/atdb`)"
+      - "traefik.http.routers.atdb-backend.rule=Host(`sdc-dev.astron.nl`) && PathPrefix(`/atdb`)"
       - "traefik.http.services.atdb-backend.loadbalancer.server.port=8000"
 
     depends_on:

atdb/docker-compose.yml → atdb/docker/docker-compose-dev.yml

@@ -40,7 +40,7 @@ services:
       - "traefik.enable=true"
       - "traefik.http.routers.atdb-backend.entryPoints=atdb-ldv"
       - "traefik.http.routers.atdb-backend.service=atdb-backend"
-      - "traefik.http.routers.atdb-backend.rule=Host(`sdc.astron.nl`) && PathPrefix(`/atdb`)"
+      - "traefik.http.routers.atdb-backend.rule=Host(`sdc-dev.astron.nl`) && PathPrefix(`/atdb`)"
       - "traefik.http.services.atdb-backend.loadbalancer.server.port=8000"
 
     depends_on: