|
|
|
[[_TOC_]]
|
|
|
|
|
|
|
|
## ESAP Architecture & Implementation
|
|
|
|
|
|
|
|
This is the implementation documentation of ESAP.
|
|
|
|
Mainly showing the ESAP API Gateway, but also some of the ESAP GUI that interfaces with the API.
|
|
|
|
|
|
|
|
### Current situation
|
|
|
|
|
|
|
|
![ESAP_Architecture-Page-1.drawio](uploads/1d83b0a3797163d7243f546c35d73968/ESAP_Architecture-Page-1.drawio.png)
|
|
|
|
|
|
|
|
## ESAP API Gateway
|
|
|
|
|
|
|
|
*ESAP API Gateway* is a 'backend' web application written in [**Django**](https://docs.djangoproject.com/en/3.0/).
|
|
|
|
The API Gateway provides a range of services that can be accessed through a REST API provided by the [**Django Rest Framework**](https://www.django-rest-framework.org/).
|
|
|
|
|
|
|
|
|
|
|
|
### Configuration
|
|
|
|
ESAP is configurable and expandable.
|
|
|
|
|
|
|
|
Part of the configuration is stored in lightweight **sqlite** databases which are natively supported by **Django** and do not require the installation and use of external database servers.
|
|
|
|
This part of the configuration is mostly for static information about *archives*, *catalogs*, *datasets* and query *parameters*.
|
|
|
|
These settings can be edited through the Django 'admin' app (see bottom of the page)
|
|
|
|
|
|
|
|
The GUI configuration is stored in python files in the 'backend' to keep the application flexibile and prevent the need for 'business logic' knowledge in the frontend (GUI).
|
|
|
|
|
|
|
|
For more details see:
|
|
|
|
* [ESAP Configuration](ESAP-configuration).
|
|
|
|
|
|
|
|
### Databases
|
|
|
|
The ESAP API gateway has a **sqlite** (configuration) database per app (query, rucio, ida, ...).
|
|
|
|
The contents of these databases are accessable through the REST API.
|
|
|
|
|
|
|
|
To be able to deploy ESAP in different configurations,
|
|
|
|
these database are deployed outside of ESAP itself in a location accessable by the backend.
|
|
|
|
For example as a mounted docker volume in shared directory.
|
|
|
|
|
|
|
|
For more details see:
|
|
|
|
* [ESAP Databases](ESAP-databases).
|
|
|
|
|
|
|
|
### Service Categories
|
|
|
|
The services are organised in *service categories*, like *query* (data discovery), *rucio* and *ida*.
|
|
|
|
There is a Django 'app' for each *service category*.
|
|
|
|
All services within a *service category* have the same API.
|
|
|
|
|
|
|
|
For example, all query services (like VO, ALTA, rucio, etc) can be accessed by the `esap-api/query/query` endpoint and
|
|
|
|
must therefore all implement the `construct_query` and `run_query` functionality in
|
|
|
|
their *service_connectors*.
|
|
|
|
|
|
|
|
For more details per *service category* see:
|
|
|
|
* [query](Service-Category---Query)
|
|
|
|
* [rucio](Service-Category:-Rucio) (not yet documented)
|
|
|
|
* [ida](Service-Category---Interactive-Analyses) (not yet documented)
|
|
|
|
|
|
|
|
### Services & Service Connectors
|
|
|
|
A *service* is the implementation of a *service category*.
|
|
|
|
The python file where a *service* is implemented is called the *service connector* and the configuration database points to this file per archive/dataset.
|
|
|
|
The API for all services within a *service category* is similar,
|
|
|
|
but the business logic can vary per archive, depending on the specific institute or 'use case').
|
|
|
|
The results are returned as a json response to the REST API.
|
|
|
|
|
|
|
|
### OIDC, IAM and federated AAI
|
|
|
|
Both the ESAP application and our institutes are registered at ESCAPE IAM.
|
|
|
|
ESAP API Gateway can contact IAM to get a token, and share that with the GUI through the session.
|
|
|
|
|
|
|
|
For more details see:
|
|
|
|
* [OIDC,-IAM-and-federated-AAI](OIDC, IAM and federated AAI)
|
|
|
|
|
|
|
|
|
|
|
|
### Currently deployed version of ESAP at ASTRON:
|
|
|
|
|
|
|
|
These are the currently deployed (latest) versions of the GUI and API Gateway.
|
|
|
|
The API Gateway is accessed by the GUI, and not directly by the end user.
|
|
|
|
|
|
|
|
* [ESAP GUI](https://sdc-dev.astron.nl/esap-gui/)
|
|
|
|
* [ESAP API Gateway](https://sdc-dev.astron.nl/esap-api/query/)
|
|
|
|
|
|
|
|
![esap_gui](uploads/b911abe312b413f32cd4b12a38a5bd1b/esap_gui.PNG)
|
|
|
|
|
|
|
|
|
|
|
|
## Admin Application
|
|
|
|
Django comes with an Admin application. You need an account with admin privileges to edit the databases.
|
|
|
|
This is an example of the admin app for the ASTRON instance of ESAP:
|
|
|
|
|
|
|
|
* https://sdc-dev.astron.nl/esap-api/admin/. You need an account with admin privileges for that.
|
|
|
|
|
|
|
|
|
|
|
|
![esap_admin](uploads/ccb326b4141d34b1e9555da21fbb847d/esap_admin.PNG)
|
|
|
|
|