Create home authored by Manuel Parra's avatar Manuel Parra
[[_TOC_]]
## ESAP Architecture & Implementation
This is the implementation documentation of ESAP.
Mainly showing the ESAP API Gateway, but also some of the ESAP GUI that interfaces with the API.
### Current situation
![ESAP_Architecture-Page-1.drawio](uploads/1d83b0a3797163d7243f546c35d73968/ESAP_Architecture-Page-1.drawio.png)
## ESAP API Gateway
*ESAP API Gateway* is a 'backend' web application written in [**Django**](https://docs.djangoproject.com/en/3.0/).
The API Gateway provides a range of services that can be accessed through a REST API provided by the [**Django Rest Framework**](https://www.django-rest-framework.org/).
### Configuration
ESAP is configurable and expandable.
Part of the configuration is stored in lightweight **sqlite** databases which are natively supported by **Django** and do not require the installation and use of external database servers.
This part of the configuration is mostly for static information about *archives*, *catalogs*, *datasets* and query *parameters*.
These settings can be edited through the Django 'admin' app (see bottom of the page)
The GUI configuration is stored in python files in the 'backend' to keep the application flexibile and prevent the need for 'business logic' knowledge in the frontend (GUI).
For more details see:
* [ESAP Configuration](ESAP-configuration).
### Databases
The ESAP API gateway has a **sqlite** (configuration) database per app (query, rucio, ida, ...).
The contents of these databases are accessable through the REST API.
To be able to deploy ESAP in different configurations,
these database are deployed outside of ESAP itself in a location accessable by the backend.
For example as a mounted docker volume in shared directory.
For more details see:
* [ESAP Databases](ESAP-databases).
### Service Categories
The services are organised in *service categories*, like *query* (data discovery), *rucio* and *ida*.
There is a Django 'app' for each *service category*.
All services within a *service category* have the same API.
For example, all query services (like VO, ALTA, rucio, etc) can be accessed by the `esap-api/query/query` endpoint and
must therefore all implement the `construct_query` and `run_query` functionality in
their *service_connectors*.
For more details per *service category* see:
* [query](Service-Category---Query)
* [rucio](Service-Category:-Rucio) (not yet documented)
* [ida](Service-Category---Interactive-Analyses) (not yet documented)
### Services & Service Connectors
A *service* is the implementation of a *service category*.
The python file where a *service* is implemented is called the *service connector* and the configuration database points to this file per archive/dataset.
The API for all services within a *service category* is similar,
but the business logic can vary per archive, depending on the specific institute or 'use case').
The results are returned as a json response to the REST API.
### OIDC, IAM and federated AAI
Both the ESAP application and our institutes are registered at ESCAPE IAM.
ESAP API Gateway can contact IAM to get a token, and share that with the GUI through the session.
For more details see:
* [OIDC,-IAM-and-federated-AAI](OIDC, IAM and federated AAI)
### Currently deployed version of ESAP at ASTRON:
These are the currently deployed (latest) versions of the GUI and API Gateway.
The API Gateway is accessed by the GUI, and not directly by the end user.
* [ESAP GUI](https://sdc-dev.astron.nl/esap-gui/)
* [ESAP API Gateway](https://sdc-dev.astron.nl/esap-api/query/)
![esap_gui](uploads/b911abe312b413f32cd4b12a38a5bd1b/esap_gui.PNG)
## Admin Application
Django comes with an Admin application. You need an account with admin privileges to edit the databases.
This is an example of the admin app for the ASTRON instance of ESAP:
* https://sdc-dev.astron.nl/esap-api/admin/. You need an account with admin privileges for that.
![esap_admin](uploads/ccb326b4141d34b1e9555da21fbb847d/esap_admin.PNG)