From 6b5212b479f16b5d8939c04e336ec331203ad615 Mon Sep 17 00:00:00 2001
From: John Swinbank <swinbank@astron.nl>
Date: Fri, 13 Jan 2023 19:25:21 +0100
Subject: [PATCH] Respond to feedback from Gareth Hughes
---
contents/1-intro.tex | 4 ++--
contents/2-vision.tex | 11 +++++------
contents/3-delivered.tex | 2 +-
contents/5-esfri.tex | 4 ++--
contents/6-future.tex | 2 ++
main.tex | 2 +-
6 files changed, 13 insertions(+), 12 deletions(-)
diff --git a/contents/1-intro.tex b/contents/1-intro.tex
index f8d1bf7..dbd2681 100644
--- a/contents/1-intro.tex
+++ b/contents/1-intro.tex
@@ -8,7 +8,7 @@ A fundamentally web-based application, it serves a key part of the interface bet
At the time of writing, the funded \gls{ESCAPE} project is reaching its conclusion.
Substantial development effort has been invested into \pgls{ESAP} and the surrounding ecosystem of software and services.
-At the same time, we do not believe that the development of \pgls{ESAP} itself is coming to an end: the technologies developed, and the workflows pioneered, have shown real potential for widespread future application.
+However, we do not believe that the development of \pgls{ESAP} itself is coming to an end: the technologies developed, and the workflows pioneered, have shown real potential for widespread future application.
This makes it an opportune moment to step back and assess the current state of and future plans for \pgls{ESAP}.
What have we achieved within the context of \gls{ESCAPE} so far?
How can we build upon these achievements in the future?
@@ -36,4 +36,4 @@ In \cref{sec:vision} we the overall vision for the \pgls{ESAP} system: what did
The second part of this document moves on to asses how successful that vision and implementation have been in addressing user --- and in particular \gls{ESFRI} --- needs.
In particular, \cref{sec:requirements} tabulates the documented requirements and compares them to delivered product, while \cref{sec:esfri} describes how some of the major \gls{ESCAPE} project \gls{ESFRI}s are engaging with and building upon the work carried out by \gls{WP}5.
-Finally, we consider potential future enhancements in \cref{sec:future}, discuss the availability of \gls{ESAP} and its integration with \gls{EOSC} in \cref{sec:access}, and provide a summary of the results of this analysis is presented in \cref{sec:conclusions}.
+Finally, we consider potential future enhancements in \cref{sec:future}, discuss the availability of \gls{ESAP} and its integration with \gls{EOSC} in \cref{sec:access}, and provide a summary of the results of this analysis in \cref{sec:conclusions}.
diff --git a/contents/2-vision.tex b/contents/2-vision.tex
index 575ddd3..c146157 100644
--- a/contents/2-vision.tex
+++ b/contents/2-vision.tex
@@ -17,7 +17,7 @@ By \emph{science platform}, we mean an interactive web-based environment which s
By \emph{toolkit}, we mean that \pgls{ESAP} is not, in itself, intended to be deployed independently.
Rather, \pgls{ESAP} provides a collection of software components, design patterns, and best practices that enable \glspl{ESFRI} or other groups that have a library of bespoke services (for example data archives, or interactive notebook environments) to make them available to a user community in a coherent, consistent, and integrated way.
-Using the \pgls{ESAP} toolkit, \gls{ESCAPE} project partners can make assist users in engaging with the services provided in the other ESCAPE work packages by:
+Using the \pgls{ESAP} toolkit, \gls{ESCAPE} project partners provide assistance to users in engaging with the services provided in the other ESCAPE work packages by:
\begin{itemize}
@@ -102,7 +102,7 @@ The current \pgls{ESAP} system provides no concept of federation between instanc
\subsubsection{User Interface}
\label{sec:vision:capabilities:ui}
-\gls{ESAP} is primarily a web application: the central hub (the “\gls{API} Gateway”, or “back-end”) runs on one or more servers, and users interact with it by making \gls{HTTP} requests.
+\gls{ESAP} is primarily a web application: the central hub (the “\gls{API} Gateway”) runs on one or more servers, and users interact with it by making \gls{HTTP} requests.
The \gls{ESCAPE} project delivery includes a customizable front-end application (“ESAP-GUI”) which runs in the browser and communicates with the back-end.
This separation of concerns is illustrated in \cref{fig:vision:capabilities:ui}.
In principle, it is possible to support alternative \glspl{GUI} which communicate with the same back-end, although none have yet been implemented.
@@ -124,8 +124,7 @@ Users may be asked to log in to access \pgls{ESAP} itself, or to use some or all
This step is not required: if both the owner of the \pgls{ESAP} instance and the owner of any services being accessed make them available to the general public, then \pgls{ESAP} need not force the user to log in.
In general, however, users are expected to log in before using the data orchestration services (\cref{sec:vision:capabilities:orch}).
-\pgls{ESAP} as delivered by this work package supports user authentication through the \gls{ESCAPE} \gls{IAM} service\footnote{\url{https://iam-escape.cloud.cnaf.infn.it/login}}.
-Where possible, \pgls{ESAP} is designed to be flexible and adaptable to other systems, but explicit support for other systems is outside the scope of this work package.
+\pgls{ESAP} as delivered by this work package is designed to support authentication through any \gls{OIDC}-compliant system, although it has only been extensively tested and integrated with the \gls{ESCAPE} \gls{IAM} service\footnote{\url{https://iam-escape.cloud.cnaf.infn.it/login}}.
\subsubsection{Data Orchestration within \glsentrytext{ESAP}}
\label{sec:vision:capabilities:orch}
@@ -185,7 +184,7 @@ Batch data processing describes a situation which is in many ways similar to \gl
\item{the user does not interact with the computing systems while processing takes place;}
-\item{processing generally happens at scale, perhaps being distributed over multiple computing systems.}
+\item{processing generally happens at scale, with workflows operating in parallel and often being distributed over multiple computing systems.}
\end{itemize}
@@ -202,7 +201,7 @@ Batch data processing describes a situation which is in many ways similar to \gl
\subsubsection{Service and Software Discovery}
\label{sec:vision:capabilities:discovery}
-\pgls{ESAP} provides deep integration with the \gls{OSSR}, and other repositories of software and services if appropriate.
+\pgls{ESAP} makes use of the eOSSR library developed in \gls{WP}3 \autocite{vuillaume_thomas_2022_6826881} to provide deep integration with the \gls{OSSR}, and could be adapted to use other software repositories where necessary.
This will make it possible for users to discover capabilities which are of relevance to them.
In particular, \pgls{ESAP} helps users discover software workflows and compute and storage infrastructure that can be used to execute both \gls{IDA} and batch processing tasks (as described in \cref{sec:vision:capabilities:ida,sec:vision:capabilities:batch}).
diff --git a/contents/3-delivered.tex b/contents/3-delivered.tex
index 93b48a4..11bb93a 100644
--- a/contents/3-delivered.tex
+++ b/contents/3-delivered.tex
@@ -170,7 +170,7 @@ Although the current \gls{IDA} system focuses on Jupyter notebooks, we expect to
\subsection{Batch Processing}
\label{sec:delivered:batch}
-\pgls{ESAP} builds on the asynchronous job management system described in \cref{sec:delivered:gateway} to provide a batch computing environment that is capable of launching jobs to a DIRAC \autocite{dirac:2022} environment, monitoring their progress, and retrieving their results.
+\pgls{ESAP} builds on the asynchronous job management system described in \cref{sec:delivered:gateway} to provide a batch computing environment that is capable of launching jobs to a DIRAC \autocite{dirac:2022} environment, monitoring their progress.
This system currently works with a limited number of pre-defined batch processing workflows.
Future work (\cref{sec:future:batch}) may include extension to a more flexible job definition system and integration with additional batch processing middleware.
diff --git a/contents/5-esfri.tex b/contents/5-esfri.tex
index e66b7b3..5731a42 100644
--- a/contents/5-esfri.tex
+++ b/contents/5-esfri.tex
@@ -21,7 +21,7 @@ It is currently under construction at two sites --- the \gls{CTAO} Northern Arra
\gls{CTAO} will consist of three classes of telescopes, covering a broad energy range from 20\,GeV to 300\,TeV the Large-Sized Telescope (LST), the Medium-Sized Telescope (MST) and the Small-Sized Telescope (SST).
During the Construction Phase, the plan for CTAO Northern Array includes 4 LSTs and 9 MSTs, whilst the CTAO Southern Array will feature 14 MSTs and 37 SSTs.
-\gls{CTAO} is expected to drive a five- to ten-fold increase in the number of known astronomical gamma ray sources, potentially detecting as many as 1,000 new previously-unknown objects.
+\gls{CTAO} is expected to drive a five- to ten-fold increase in the number of known astronomical gamma-ray sources, potentially detecting as many as 1,000 new previously-unknown objects.
It is particularly suited to observing short (less than a minute) timescale phenomena, making it a key instrument in the future multi-wavelength and multi-messenger infrastructure of time domain astronomy.
The primary scientific drivers of \gls{CTAO} are:
@@ -50,7 +50,7 @@ Specifically, \gls{CTAO} looks to \gls{ESAP} as the potential basis for a scienc
The \gls{CTAO} team identify \gls{ESAP}'s modular and extensible nature --- described in \cref{sec:vision:extensibility} --- as important to their needs: it makes it possible to extend the platform to add new services and archives by building on \gls{REST} interfaces.
This helps enable the identified \gls{CTAO} goals of common access mechanisms.
-The project has already put this into practice, for example by linking \pgls{ESAP} with the \gls{OSSR}, as described in \cref{sec:delivered:ida} using the eOSSR software developed in \gls{WP}3 \autocite{vuillaume_thomas_2022_6826881}.
+The project has already put this into practice, for example by linking \pgls{ESAP} with the \gls{OSSR}, as described in \cref{sec:delivered:ida}.
\paragraph{Analysis Workflows}
diff --git a/contents/6-future.tex b/contents/6-future.tex
index 949c078..3e3369f 100644
--- a/contents/6-future.tex
+++ b/contents/6-future.tex
@@ -77,6 +77,8 @@ There are numerous improvements that could be made here, including:
\end{itemize}
+Further, we note that the DIRAC system itself is under active development and there will soon be opportunities for \gls{ESAP} to take advantage of its new capabilities including token-based authorization and a \gls{REST} \gls{API}.
+
\subsubsection{Advanced Metadata for Job and Infrastructure Selection}
\label{sec:future:metadata}
diff --git a/main.tex b/main.tex
index a64c5ee..cc33f49 100644
--- a/main.tex
+++ b/main.tex
@@ -9,7 +9,7 @@
\setDocTitle{Final Assessment of the Performance of the Science Platform Prototype}
\setWorkPackage{WP5, ESFRI Science Analysis Platform}
\setLeadAuthor{John D. Swinbank (ASTRON)}
-\setOtherAuthors{James Collinson (SKAO), Stelios Voutsinas (U.~Edinburgh)}
+\setOtherAuthors{James Collinson (SKAO), Gareth Hughes (CTAO), Stelios Voutsinas (U.~Edinburgh)}
\setDueDate{2022-11-30}
\setDueMonth{M46}
--
GitLab