LDV Specification

LDV Specification Application. For filling ATDB-LDV with processing tasks for LOFAR data

Documentation (Confluence)

• The plan: https://support.astron.nl/confluence/pages/viewpage.action?pageId=84215267

• https://support.astron.nl/confluence/display/SDCP/LDV+Documentation

• deployment diagram of the current situation (in production)

  • https://drive.google.com/file/d/1_j9Fp505pZTxcmzAEdgftdPkoIFrKfAX/view?usp=sharing

Collaborate

• create your branch from main

• add your functionality

• test your functionality locally

• merge main into your branch before creating a MR

• merge with main

• deploy in test, and test it

• deploy in production, and test it

Local update

After a collegue has made changes, then locally:

  > git pull
  > pip install -r requirements\dev.txt
  > python manage.py migrate --settings=ldvspec.settings.dev  

Local Development Environment

Postgres Database in Docker

Run docker-compose -f docker-compose-postgres-dev.yml up -d with the following compose file to spin up a new Postgres container. See the docker-compose-postgres-dev.yml file in the docker directory. (not that port 5433 is used. You can change that at will, but then also change it in dev.py)

version: "3.7"
services:
      
  ldv-spec-db:
    image: postgres:14
    container_name: ldv-spec-postgres
    expose:
      - 5433
    ports:
      - 5433:5432
    environment:
      POSTGRES_PASSWORD: "secret"
      POSTGRES_USER: "postgres"
      POSTGRES_DB: "ldv-spec-db"      
    volumes:
      - ldv-spec-db:/var/lib/postgresql/data
    restart: always
    
volumes:
  ldv-spec-db:

Django Application

• clone the repo

• open the project in Pycharm

• create a venv (File -> Settings -> Project -> Project Interpreter -> (click cog) -> add)

• pip install -r requirements\dev.txt

• check and/or change the database connection in settings/dev/py. In this example it connects to a database server on 'raspiastro', you have to change that to the server where you run your Postgres Docker container (localhost?)

DATABASES = {
    'default': {
         'ENGINE': 'django.db.backends.postgresql_psycopg2',
         'USER': 'postgres',
         'PASSWORD': 'secret',
         'NAME': 'ldv-spec-db',
         'HOST': 'raspiastro',
         'PORT': '5433',
    },
}

   > python manage.py migrate --settings=ldvspec.settings.dev
   > python manage.py runserver --settings=ldvspec.settings.dev
   > python manage.py createsuperuser --settings=ldvspec.settings.dev

Test Environment

• https://sdc-dev.astron.nl/ldvspec/

Production Environment

• https://sdc.astron.nl/ldvspec/

Configuration

See ~/shared/ldvspec.env for database credentials and where to find ATDB-LDV

NOTE: currently a postgres database in a Docker container is also used in production. This will change to a database on the sdc-db machine.

admin user

• admin:admin

Build & Deploy

The CI/CD pipeline creates 2 Docker containers:

• ldv-specification : The Django application
• ldv-spec-postgres : The Postgres database

The database can also be accessed externally:

• host : sdc-dev.astron.nl / sdc.astron.nl
• port : 12000
• database: ldv-spec-db

Manual steps (add them somewhere)

Log into the ldv-specification container. (using the portainer GUI or with the docker exec)

• https://sdc-dev.astron.nl/portainer/
• https://sdc.astron.nl/portainer/#/home

> cd /src 
> python manage.py migrate --settings=ldvspec.settings.docker_sdc
> python manage.py createsuperuser --settings=ldvspec.settings.docker_sdc