diff --git a/contents/2-vision.tex b/contents/2-vision.tex index f350798aeedff8d9b7c5b61b32650610149d1140..81566453e3712cbf9f54bab9f0c27e134105dc7a 100644 --- a/contents/2-vision.tex +++ b/contents/2-vision.tex @@ -40,7 +40,7 @@ Using the \pgls{ESAP} toolkit, \gls{ESCAPE} project partners can make assist use This relationship between \pgls{ESAP} and the capabilities exposed by the other \gls{ESCAPE} work packages --- and the wider service infrastructure within which it exists --- is illustrated in \cref{fig:vision:interfaces} and explored further in subsequent sections. -By design, \pgls{ESAP} is extensible: rather than attempting to anticipate every possible type of data repository, software, compute system, or other service provider, the platform will provide generic interfaces through which it can be extended to encompass new functionality. +By design, \pgls{ESAP} is extensible: rather than attempting to anticipate every possible type of data repository, software, compute system, or other service provider, the platform provides generic interfaces through which it can be extended to encompass new functionality. In short, our approach is not to attempt to provide a single, integrated platform to which all researchers must adapt, but rather a set of functionalities from which various communities and research infrastructures can assemble an analysis platform geared to their specific needs. Deploying such a science platform provides at the scale of a system like \pgls{EOSC} provides a natural opportunity to integrate with the data and computing fabric this environment encompasses while simultaneously accessing the tools, techniques, and expertise other research domains bring to that environment. @@ -102,10 +102,10 @@ The current \pgls{ESAP} system provides no concept of federation between instanc \label{sec:vision:capabilities:ui} \gls{ESAP} is primarily a web application: the central hub (the “\gls{API} Gateway”, or “back-end”) runs on one or more servers, and users interact with it by making \gls{HTTP} requests. -The work package will provide a customizable front-end application (“ESAP-GUI”) which runs in the browser and communicates with the back-end. +The \gls{ESCAPE} project delivery includes a customizable front-end application (“ESAP-GUI”) which runs in the browser and communicates with the back-end. This separation of concerns is illustrated in \cref{fig:vision:capabilities:ui}. -In principle, it may be possible to support alternative \glspl{GUI} which communicate with the same back-end. -Providing such alternatives is out of scope for this work package, but provides scope for future extension of the work if appropriate. +In principle, it is possible to support alternative \glspl{GUI} which communicate with the same back-end, although none have yet been implemented. +Providing such alternatives is out of scope for the \gls{ESCAPE} project, but provides scope for future extension of the work if appropriate. \begin{figure} \begin{center} @@ -123,19 +123,19 @@ Users may be asked to log in to access \pgls{ESAP} itself, or to use some or all This step is not required: if both the owner of the \pgls{ESAP} instance and the owner of any services being accessed make them available to the general public, then \pgls{ESAP} need not force the user to log in. In general, however, users are expected to log in before using the data orchestration services (\cref{sec:vision:capabilities:orch}). -\pgls{ESAP} as delivered by this work package will provide for user authentication through the \gls{ESCAPE} \gls{IAM} service\footnote{\url{https://iam-escape.cloud.cnaf.infn.it/login}}. +\pgls{ESAP} as delivered by this work package supports user authentication through the \gls{ESCAPE} \gls{IAM} service\footnote{\url{https://iam-escape.cloud.cnaf.infn.it/login}}. Where possible, \pgls{ESAP} is designed to be flexible and adaptable to other systems, but explicit support for other systems is outside the scope of this work package. \subsubsection{Data Orchestration within \glsentrytext{ESAP}} \label{sec:vision:capabilities:orch} -The fundamental \pgls{ESAP} workflow is that the user will query one or more archives to identify data of interest, then dispatch that data to \gls{IDA} or bulk processing systems for processing. +The fundamental \pgls{ESAP} workflow is that the user will query one or more archives to identify data of interest, then dispatch that data to \gls{IDA} or bulk processing systems for processing, collecting results and forwarding them to additional workflow stages as necessary. To support this model, \pgls{ESAP} maintains a per-user list of active data items: the “shopping basket”. This basket is persistent: (a representation of) the data the user has selected is serialized as \gls{JSON}, and the results are stored in a database. Note that the basket is not generally expected to contain a complete representation of the data in question (it will not store multi-\gls{GB} images or query results), but rather it will contain sufficient metadata that the data can be fetched and manipulated on demand (for example, it will store the query which produces the result in question, or a path or other identifier which enables data to be fetched from the “data lake” or other storage). -Services integrated with the \pgls{ESAP} system will be able to edit, augment, and update the contents of the users' shopping basket. +Services integrated with the \pgls{ESAP} system are able to edit, augment, and update the contents of the users' shopping basket. This shopping basket metaphor extends include services --- such as \gls{IDA} or batch compute facilities --- and workflows from the \pgls{OSSR} and other repositories: as they move through the system, users will be able to identify services or software of interest, and store them for use later. @@ -171,7 +171,7 @@ It is therefore essential that \gls{ESAP} implement a flexible capability for in The architecture described in \cref{sec:vision:capabilities:ui}, together with the data orchestration system described in \cref{sec:vision:capabilities:orch}, are designed to make this possible. Specifically, \pgls{ESAP} provides an \glspl{API} through which \gls{IDA} systems can access the “shopping basket”, both to retrieve data items and to provide (appropriately authenticated) updates from the \gls{IDA} system as the user saves their analysis. -The expectation is that the \gls{IDA} system will write substantial data products (such as output images) to bulk storage (such as the \gls{DIOS} data lake), and return references to them to \pgls{ESAP} for further analysis. +The expectation is that the \gls{IDA} system will write substantial data products (such as output images) to bulk storage (such as the \gls{DIOS} data lake), and return references to them to \pgls{ESAP} for further analysis, but this can be adapted for specific project requirements. \subsubsection{Batch Data Processing} \label{sec:vision:capabilities:batch} @@ -248,18 +248,4 @@ Instead, the team has focused on: \end{itemize} -The detailed list of service integrations which will be supplied in the core \pgls{ESAP} delivery by \gls{ESCAPE} \gls{WP}5 is still under development. -However, we expect to provide at least: - -\begin{itemize} - -\item{data query and data discovery based on major \gls{ESFRI} archives;} -\item{integration with Rucio-based \autocite{rucio:2019} data lake systems;} -\item{\gls{VO} query capabilities;} -\item{\gls{SAMP} integration;} -\item{integration with Jupyter-based \gls{IDA} facilities, probably based around BinderHub \autocite{binder:2018} and/or Rosetta\footnote{\url{https://rosetta.oats.inaf.it/main/}; \url{https://github.com/sarusso/Rosetta}};} -\item{integration with at least one batch computing service, probably through DIRAC \autocite{dirac:2018}.} - -\end{itemize} - -The results of these activities are described in \cref{sec:current}. +The services which are supported at the completion of the \gls{ESCAPE} project are described in \cref{sec:current}, but furthers will be added by the future \pgls{ESAP} open source project (described in \cref{sec:future}). diff --git a/contents/7-future.tex b/contents/7-future.tex index 1bd1f060efbf2ae29644b987fca8690e1ad01d71..9bf0bdc5b9fb404cd31aa55d5f8ec52c417e4128 100644 --- a/contents/7-future.tex +++ b/contents/7-future.tex @@ -1,5 +1,8 @@ \section{Future Prospects} \label{sec:future} +\subsection{The ESAP Open Source Project} +\label{sec:future:osp} + \subsection{Federation} \label{sec:future:fed}