% documentclass{} takes the following mutually-exclusive options:
%
% note -- formats the document as a technical note.
% proposed -- formats the document as a proposed change to the baseline.
% baseline -- formats the document as part of the technical baseline.
\documentclass[note]{astron}
\documentclass[note]{astron}
\input{meta}
\input{meta}
\input{changes}
\input{changes}
\setDocTitle{ASTRON SDC Document Example}
\setDocTitle{SDC Program Activities During 2021}
\setDocProgram{SDC}
\setDocProgram{SDC}
\setDocAuthors{
\setDocAuthors{
\addPerson{John D. Swinbank}{ASTRON}{\vcsDate}
\addPerson{John D. Swinbank}{ASTRON}{\vcsDate}
...
@@ -21,34 +16,77 @@
...
@@ -21,34 +16,77 @@
\begin{document}
\begin{document}
\maketitle
%\maketitle
\printglossary[title=List of Abbreviations]
%\printglossary[title=List of Abbreviations]
\clearpage
%\clearpage
\section{Getting Started}
\section*{\docTitle}
\label{sec:getting-started}
\begin{enumerate}
\subsection*{Science Data Centre Context \& Governance}
\item{Edit \verb|Makefile| and set \verb|DOCHANDLE| to the appropriate handle for your document.
The Science Data Centre is a new facility which aims to be the single point of contact for users of instrumentation operated by ASTRON.
Refer to the \href{https://support.astron.nl/confluence/display/SDCP/List+of+SDC-Series+Documents}{list of SDC series documents} which have been issued to date, and choose the next one in sequence.}
It will ultimately provide services including --- but not limited to --- proposal preparation and submission; archiving and providing access to data; generation of science-ready data products; user-friendly interfaces to discover and access data; and on-line interactive analysis facilities.
\item{Check the preamble of this file to ensure that the document title, programme, and author(s) have been set correctly.}
As an facility, the SDC is run by an operational unit within ASTRON's Astronomy \& Operations department: the SDC Operations team.
The facility is developed and supported by the SDC Development Program, which draws on staff from across the institute.
\item{Insert your text here.}
The SDC was formed in 2020, inheriting a complex service portfolio from its predecessors within ASTRON.
During 2021, our activities have included both establishing the organizational structures on which the SDC will be built and managed, on a number of high-impact technical deliverables, and on engagement with a number of major international projects.
\item{Use the \verb|\gls| command to refer to abbreviations, like \gls{SDC}.}
\subsection*{Scaled Agile Framework}
\item{Use the \verb|\autocite| command to cite other documents, like \autocite{SDC-006}.}
The SDC Program has adopted an organization based on the \emph{Scaled Agile Framework}\footnote{\url{https://scaledagileframework.com}}, or SAFe.
The SAFe system was chosen because it provides best-practice structures for organizing agile software development at scale in a way which focuses both on quality and on responsiveness to customer needs.
Further, SAFe has been shown to be very successful in the scientific context by our peers and colleagues in the Square Kilometre Array project\footnote{\url{https://www.skatelescope.org/}}.
\item{Use the \verb|\cref| command to refer to sections, tables, or other material in this document, like \cref{sec:getting-started}.}
Adopting SAFe has required a substantial effort in training and adapting our ways of working during 2021, but initial results have been very positive and the SDC Program enters 2022 with a solid organizational structure and confidence that we are ready to tackle the challenges of SDC construction!
\item{Run \verb|make| to compile the document.}
\subsection*{Pipeline Execution Framework}
\end{enumerate}
During early 2021, the SDC Program delivered the \emph{ATDB-LDV} pipeline execution framework to operations.
\emph{ATDB-LDV} provides a standardized systems for processing data within the operational SDC facility.
Throughout 2021, it has been used by the \emph{LOFAR Data Valorisation} project in support of ongoing processing campaigns.
We expect that development of the framework will continue, and further pipelines will be integrated with it, during 2022.
\clearpage
\subsection*{Prefactor Pipeline}
\printbibliography
\emph{Prefactor} is a pipeline for direction-independent calibration and imaging of LOFAR data.
It is the result of several years of development effort, both within ASTRON and across the broader community of LOFAR users, and draws on a number of standard software components developed by ASTRON's Smart Backend group and its partners.
During 2021, the SDC team accepted responsibility for continued development and maintenance of Prefactor.
It has been updated so that it is now defined using the Common Workflow Language\footnote{\url{https://www.commonwl.org}}, which forms the basis of the pipeline execution system used by the SDC facility.
This updated Prefactor pipeline was made available in late 2021, and, at time of writing, is undergoing integration with the \emph{ATDB-LDV} execution framework.
During 2022, it will form the basis of processing carried out by the \emph{LOFAR Data Valorisation} project.
\subsection*{SKA Regional Centres}
When it becomes operational in the second half of this decade, the Square Kilometre Array will generate enormous quantities of data.
The SKA Observatory itself will not make this data directly available to end users.
Rather, it will be supplied to a globally-distributed network of \emph{SKA Regional Centres} which are responsible for archiving the data and supporting end-user science activities.
Construction of this SRC network is a major international effort, involving stakeholders and technical teams from across the various members of the SKA.
The SDC Program, in collaboration with the SDCO Operations team and ASTRON's SKA-SDC Focus Group, are leading ASTRON's contribution to the SRC design effort.
During 2021, this has involved collecting system requirements and developing initial architectural concepts.
This work will continue into 2022, as we move towards a large-scale prototyping effort.
\subsection*{Science Analysis Platform}
The ESCAPE Project\footnote{European Science Cluster of Astronomy \& Particle Physics ESFRI Research Infrastructures; \url{https://www.projectescape.eu}} is developing common digital infrastructure for use by major research infrastructure projects across Europe.
The SDC Program leads ESCAPE's Work Package 5, which is developing a science analysis platform framework.
This provides data discovery, access, and analysis services across a wide range of science cases and communities.
During 2021, development of this system continued rapidly, reaching a culmination in an international workshop in August which demonstrated an impressive range of functionality and received positive feedback from the community.
A test version of the platform is has been made available online available\footnote{\url{https://sdc-dev.astron.nl/esap-gui/}, but note that this is strictly a development system and availability and functionality is not guaranteed.}, and work is now underway to develop an ASTRON-specific version which will be deployed within the operational SDC facility (known as ADEX, the ASTRON Data Explorer).
\subsection*{LoTSS Data Release 2 and Scientific Data Repositories}
LoTSS, the LOFAR Two-metre Sky Survey, is a major wide-area imaging survey carried out in the LOFAR high band.
Data processing for the survey has been coordinated by the LOFAR Surveys Key Science Project.
The SDC team has coordinated hosting and publication of the second data release through a scientific data repository hosted at SURF (although at time of writing the data is embargoed pending publication of an accompanying scientific paper).
Deployment of this form of repository at SURF is a key technological step for our SDC plans, and paves the way to our support of future data releases.
It is also a great example of cooperation between the ASTRON SDC, the scientific community using LOFAR, and SURF, the national e-infrastructure provider, a mode of operation which we regard as vital to the success of the SDC effort and the scientific productivity of the community.
