esap-general issueshttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues2021-06-22T13:18:18Zhttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/8decide between 1-to-n or n-to-n relationship user=>shopping items2021-06-22T13:18:18ZNico Vermaasdecide between 1-to-n or n-to-n relationship user=>shopping itemsOriginally the ShoppingBasket mechanism was (partly) implemented (by Hugh Dickingson) as a many-to-many relationship with a userprofile.
Expanding this to a fuller implementation (by me, Nico) where the user could also remove items, le...Originally the ShoppingBasket mechanism was (partly) implemented (by Hugh Dickingson) as a many-to-many relationship with a userprofile.
Expanding this to a fuller implementation (by me, Nico) where the user could also remove items, lead to dangling items in the shopping basket after all users had removed the shared items.
To overcome this problem, I implemented an alternative where the relationship between users and their shoppingbasket items is 1-to-many.
This alternative lives on a branch, but we need to decide (asap) if we will use this implementation or if somebody (Hugh?) solves that problem in the many-to-many implementation.
https://git.astron.nl/astron-sdc/esap-api-gateway/-/tree/esap-gateway-shoppingbasket2
Hugh noted that the many-to-many solution needs to stay in place for the `ComputeResources` and `SoftwareRepository` sections. But those sections have also been (partly) implemented in the `ida` app by Stelios, and depending on how the decision to move forward with that these sections in the `accounts` app may or may not disappear.
https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/1Nico VermaasNico Vermaashttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/43Define & Develop Facility finding service2021-07-27T09:39:34ZSteliosDefine & Develop Facility finding serviceDefine & Develop Facility finding service, for finding/selecting Compute Facilities that a user can/may want to use.
We need to define what types of calls this service will accept & what input/output is required for each.
To start with ...Define & Develop Facility finding service, for finding/selecting Compute Facilities that a user can/may want to use.
We need to define what types of calls this service will accept & what input/output is required for each.
To start with we can use mock data.
This task can be broken down into the following subtasks:
1) Define what metadata is needed to describe a Compute Facility. (Currently we have name, url, description & type)
2) Update the Django components (models.py / UI templates etc..) with additional fields
3) Modify UI page that provides searchable lists of Facilities so that it consumes metadata from the Gateway REST services (ESAP GUI task)
4) Document list of Compute Facilities and associated metadata that will be used for the first Prototype, and store in ESAP DB.
5) Develop page in ESAP GUI dedicated to Compute Facility discovery.
6) Update IDA Gateway REST services to implement additional services / changes to servic inputs based on 5)
7) Implement "Add Compute to Shopping cart" featurehttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/44Define & Develop JupyterHub finding service2021-07-27T09:39:42ZSteliosDefine & Develop JupyterHub finding serviceDefine and develop a JupyterHub Finding service. This may extend the Facility Finding service and accept requests for finding JHub services by keyword. We have not yet defined how these will be found, to start with we can use mock data.
...Define and develop a JupyterHub Finding service. This may extend the Facility Finding service and accept requests for finding JHub services by keyword. We have not yet defined how these will be found, to start with we can use mock data.
This service needs to be able to provide metadata for a given JHub service, like a list of available interpreters for example.https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/3Design & Build Workflow / Software discovery page2022-04-19T09:38:26ZSteliosDesign & Build Workflow / Software discovery pageDesign & build software & worfklow discovery page, along the lines of the sketches shown here: https://docs.google.com/presentation/d/18uxa9njdknJw73Vt2jUij78ot60JQfWwvSubOBA3KLA/edit#slide=id.g8d029ca29c_0_17Design & build software & worfklow discovery page, along the lines of the sketches shown here: https://docs.google.com/presentation/d/18uxa9njdknJw73Vt2jUij78ot60JQfWwvSubOBA3KLA/edit#slide=id.g8d029ca29c_0_17https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/2Develop API / Service to harvest list of Software & Workflows from the OSSR2021-11-30T10:50:46ZSteliosDevelop API / Service to harvest list of Software & Workflows from the OSSRIn the IDA we currently display a list of Worfklows (Notebooks) that are stored in an SQLLite database. WP3 & the OSSR have built a Software Repository that will include workflows for ESCAPE, and this issue is to investigate & develop th...In the IDA we currently display a list of Worfklows (Notebooks) that are stored in an SQLLite database. WP3 & the OSSR have built a Software Repository that will include workflows for ESCAPE, and this issue is to investigate & develop the mechanism in
ESAP for collecting this metadata for existing software & workflows, and displaying it to the user.
The service in the API that currently collects this metadata from the database can be found here:
https://git.astron.nl/astron-sdc/esap-api-gateway/-/blob/master/esap/ida/api/services/ida_controller.py
We need to extend this service so that it calls out to the REST API of the OSSR (Zenodo), collects the JSON response and formats it in a way that the GUI can read and display as a list.
For the above we need to implement a harvestor method as follows:
```python
def get_OSSR_entries(parameters: list) -> dict:
# GET request to the OSSR with optional parameters
# Format results
# Return formatted results
```SteliosStelioshttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/41Document and provide examples of querying the REST API of a JupyterHub for me...2022-04-19T09:23:43ZSteliosDocument and provide examples of querying the REST API of a JupyterHub for metadataFind and document how to query the REST API of a JupyterHub service to gather information like list of interpreters.
Include description of how authenticated access works, which may be required to get this metadata.Find and document how to query the REST API of a JupyterHub service to gather information like list of interpreters.
Include description of how authenticated access works, which may be required to get this metadata.https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/18Document if and how to spawn a Jupyter notebook with additional metadata info...2022-04-11T11:39:35ZSteliosDocument if and how to spawn a Jupyter notebook with additional metadata information in the environmentFind out and document how (if possible) to spawn a notebook with additional metadata information in the environment.
For example, if we want to spawn a notebook at a remote JHub service, with a parameter such as where the dataset is loca...Find out and document how (if possible) to spawn a notebook with additional metadata information in the environment.
For example, if we want to spawn a notebook at a remote JHub service, with a parameter such as where the dataset is located, how could this be done?https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/4Document if and how to spawn a Jupyter notebook with additional metadata info...2022-04-22T09:27:40ZSteliosDocument if and how to spawn a Jupyter notebook with additional metadata information in the environmentFind out and document how (if possible) to spawn a notebook with additional metadata information in the environment. For example, if we want to spawn a notebook at a remote JHub service, with a parameter such as where the dataset is loca...Find out and document how (if possible) to spawn a notebook with additional metadata information in the environment. For example, if we want to spawn a notebook at a remote JHub service, with a parameter such as where the dataset is located, how could this be done?https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/40Document metadata extraction options from Docker hub Repos.2021-07-27T09:38:43ZSteliosDocument metadata extraction options from Docker hub Repos.Find and document what metadata in a Docker Repository can be searched for, which will define what our Search capability providesFind and document what metadata in a Docker Repository can be searched for, which will define what our Search capability provideshttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/10Expand esap-userprofile-python-client with 'archive connector'2021-07-19T08:11:36ZNico VermaasExpand esap-userprofile-python-client with 'archive connector'ESAP knows the structure of the data of the 'archives' that it queries, this is defined in the configuration database of the 'query' app. ESAP can use this information to parse and structure the json blobs in the shopping_basket for know...ESAP knows the structure of the data of the 'archives' that it queries, this is defined in the configuration database of the 'query' app. ESAP can use this information to parse and structure the json blobs in the shopping_basket for known archives.
For unknown archives (like SAMP), the current json blobs can still be used.
Astron JIRA ticket: https://support.astron.nl/jira/browse/SDC-245MS31Nico VermaasNico Vermaashttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/9Expand esap-userprofile-python-client with authentication2021-07-02T19:32:12ZNico VermaasExpand esap-userprofile-python-client with authenticationThis client can be used in Jupyter Notebooks (or elsewhere) to retrieve data from the shopping basket for further analyses
Astron JIRA ticket:
https://support.astron.nl/jira/browse/SDC-237This client can be used in Jupyter Notebooks (or elsewhere) to retrieve data from the shopping basket for further analyses
Astron JIRA ticket:
https://support.astron.nl/jira/browse/SDC-237MS31Hugh DickinsonHugh Dickinsonhttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/24IDAContext defaults to questionable URL for JupyterLab service2021-07-27T08:27:00ZJohn SwinbankIDAContext defaults to questionable URL for JupyterLab service[`src/contexts/IDAContext.js` line 12](https://git.astron.nl/astron-sdc/esap-gui/-/blob/master/src/contexts/IDAContext.js#L12) provides a default URL for JupyterHub services corresponding to the SKAO JupyterHub system.
This URL is over-...[`src/contexts/IDAContext.js` line 12](https://git.astron.nl/astron-sdc/esap-gui/-/blob/master/src/contexts/IDAContext.js#L12) provides a default URL for JupyterHub services corresponding to the SKAO JupyterHub system.
This URL is over-written in [`src/components/services/Interactive.js` line 36](https://git.astron.nl/astron-sdc/esap-gui/-/blob/master/src/components/services/Interactive.js#L36) when the value selected by the user in the dropdown is changed.
If the user never adjusts the dropdown, then the default URL is never over-written. In this case, the `Deploy` button will redirect the user to the default set in `IDAContext.js`, regardless of what the service is actually selected in the dropdown.
That's bad enough as it stands, but what if there is only one service configured (e.g., following [esap-api-gateway!100](https://git.astron.nl/astron-sdc/esap-api-gateway/-/merge_requests/100)? In this case, the value which appears in the dropdown never changes (because there is nothing else to select), so if the default doesn't happen to match the only service configured then there's no way for the user to override it.John SwinbankJohn Swinbankhttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/25Implement Checkout page2021-07-27T08:27:38ZSteliosImplement Checkout pageAs shown here:
https://docs.google.com/presentation/d/18uxa9njdknJw73Vt2jUij78ot60JQfWwvSubOBA3KLA/edit#slide=id.g8d029ca29c_0_99As shown here:
https://docs.google.com/presentation/d/18uxa9njdknJw73Vt2jUij78ot60JQfWwvSubOBA3KLA/edit#slide=id.g8d029ca29c_0_99https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/45Implement Scheduler Service2021-07-27T09:39:55ZSteliosImplement Scheduler ServiceImplement a Scheduler Service, which given a triplet of Compute Facility, Workflow and Dataset, will spawn a job in a facility, or redirect a user to the given compute facility, depending on the type of workflow & facility,Implement a Scheduler Service, which given a triplet of Compute Facility, Workflow and Dataset, will spawn a job in a facility, or redirect a user to the given compute facility, depending on the type of workflow & facility,https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/5Implement Scheduler Service2022-04-22T09:28:01ZSteliosImplement Scheduler ServiceImplement a Scheduler (micro) Service, which given a Compute Facility, Workflow and Dataset, will spawn a job in a facility, or redirect a user to the given compute facility, depending on the type of workflow & facility,Implement a Scheduler (micro) Service, which given a Compute Facility, Workflow and Dataset, will spawn a job in a facility, or redirect a user to the given compute facility, depending on the type of workflow & facility,SteliosStelioshttps://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/42Implement Workflow finding service2021-07-27T09:39:26ZSteliosImplement Workflow finding serviceDefine & Develop Workflow finding service, for finding/selecting Notebooks and other similar Workflows. We need to define what types of calls this service will accept & what input/output is required for each.
We have not yet defined wher...Define & Develop Workflow finding service, for finding/selecting Notebooks and other similar Workflows. We need to define what types of calls this service will accept & what input/output is required for each.
We have not yet defined where this will read the worfklows from, so to start with we can use mock data.https://git.astron.nl/astron-sdc/escape-wp5/esap-general/-/issues/1Synchronize & Merge IDA Models with existing Shopping basket models2022-04-22T11:27:21ZSteliosSynchronize & Merge IDA Models with existing Shopping basket modelsIn the IDA app we have a number of models to enable the features we've outlined in our Working Group, however in some of the other apps there are similar and perhaps overlapping models / db tables which we may want to reuse, for example ...In the IDA app we have a number of models to enable the features we've outlined in our Working Group, however in some of the other apps there are similar and perhaps overlapping models / db tables which we may want to reuse, for example the "Shopping Cart".
This task is to investigate / document & make the appropriate changes to the IDA app wherever necessary so that we do not have duplicate models / tables with the same information.