This document addresses the way in which the effort to construct and operate the \gls{SDC} will be organized.
This includes both the structures which have been established within ASTRON, and the expected interactions with other stakeholders and the wider community.
\section{Structure}
\label{sec:governance:structure}
The ASTRON \gls{MT} has established two bodies to construct and operate the \gls{SDC}:
\begin{itemize}
\item{The \emph{\acrlong{SDCP}} (\acrshort{SDCP}) is responsible for developing the software and services that are necessary to operate the \gls{SDC}.}
\item{\emph{\acrlong{SDCO}} (\acrshort{SDCO}) is responsible for operating the \gls{SDC}, defining and prioritizing a service offering to meet community needs, and providing support to members of the community.}
\end{itemize}
It is expected that these bodies will draw on both other staff within ASTRON --- members of the various Competence and Focus Groups --- and from the external scientific community.
They are overseen by a Steering Committee, appointed by the ASTRON \gls{MT}.
The relationship between all of these groups is illustrated in \cref{fig:sdc-hierarchy}, while their various responsibilities are described below.
Relationships between core stakeholders in the \gls{SDC} effort.
Arrows indicate expected exchanges of information, expertise, and effort, rather than organizational hierarchy, which is discussed in the text.
}
\label{fig:sdc-hierarchy}
\end{figure}
The \gls{SDCP} is envisioned as a \emph{development programme} within ASTRON.
As such, it is formally a temporary organization which exists to develop certain defined functionality; in this case, the software required to operate the \gls{SDC}.
The \gls{SDCP} is jointly responsible with \gls{SDCO} for developing the overall architecture of the \gls{SDC} facility.
It is expected that the \gls{SDCP} will bring software development and architectural expertise to this discussion, which will be used to design the overall system based on the available infrastructure and operational needs of \gls{SDCO}.
The \gls{SDCP} is responsible for the delivery of operable \emph{software products} to the \gls{SDCO} team.
These products are combined with infrastructure --- provided by the \gls{SDCO} team --- and appropriate configuration to deploy and operate the services which comprise the \gls{SDC}.
The \gls{SDCP} has ultimate responsibility for the software used to implement all core and ancillary (but not support \autocite{SDC-006}) services deployed within the \gls{SDC} facility\footnote{It does not follow that the \gls{SDCP} must implement this software in-house; indeed, it is expected that much of the \gls{SDC} software infrastructure will be based on third-party tooling.}.
The \gls{SDCP} is further responsible for the delivery of the scientific pipelines which are used as part of regular facility operations to generate the core \glspl{SRDP}.
This includes pipelines specifically identified in \citetitle[the][]{SDC-006}\autocite{SDC-006} or other baselined documentation as well as any other pipelines which are identified as strategically important by the \gls{SDCO} team \emph{and} which are agreed as within scope by \gls{SDCP} management.
The \gls{SDCP} is expected to arrange its work to address the scientific and operational requirements, and their relative priorities, expressed by \gls{SDCO}.
Detailed scheduling of work will remain the domain of \gls{SDCP}, and it is acknowledged that development or management considerations must be accounted for in parallel with \gls{SDCO} needs.
The \gls{SDCP} is responsible for defining the working methods around the software products it delivers.
This includes, for example, adopting appropriate agile team working practices, software release methodologies and the like.
These methods will make it possible for the \gls{SDCP} to provide appropriate support to the \gls{SDCO} team when necessary.
Wherever possible, these methods will be aligned with approaches adopted across ASTRON more generally.
The \gls{SDCP} is responsible for delivering software products of high quality.
That is, the products are appropriately tested, suitably documented, and packaged adequately for distribution.
It is expected that the \gls{SDCP} will collaborate with the Software Delivery Competence Group to develop coding, testing, and packaging standards which can be applied both to the \gls{SDCP} itself and, ultimately, across ASTRON's software development effort.
The \gls{SDCP} is primarily funded and supported through executing projects which are sponsored by external organizations such as the \acrlong{EC} and \acrshort{NWO}.
The \gls{SDCP} is responsible for coordinating efforts across ASTRON to compete for and win funding in this way.
These projects have their own deliverables and milestones, and the \gls{SDCP} is responsible for ensuring that these are met and the projects are completed successfully.
Projects will be chosen, and the work will be managed, such that project deliverables contribute to the software products that \gls{SDCP} will deliver to \gls{SDCO}.
Some level of additional funding to support integration and oversight activities will also be required; this will be drawn from the ASTRON base budget and in-kind contributions as appropriate.
This structure is illustrated in \cref{fig:sdc-organization}.
According to the ``ASTRON 2.0'' organizational model, as a programme the \gls{SDCP} does not have any permanent staff.
Rather, these are sourced from Competence and Focus Groups.
Additional effort may be drawn from in-kind contributions to the \gls{ILT} or other projects, with the agreement of the \gls{ILT} Board where appropriate.
The \gls{SDCP} is responsible for coordinating with the various group leaders to ensure that the development teams are adequately resourced.
\gls{SDCP} is led by a Programme Manager, who is functionally responsible to the ASTRON \gls{MT}.
Structure of the \gls{SDC} development and operations effort.
The bulk of development effort is funded through externally-sponsored projects, with their own deliverables, shown at left.
The \gls{SDCP} integrates functionality developed for these projects and augments it with internal, ASTRON-funded work to create the \gls{SDC} Software Products.
This software is then deployed to form operational services by the \gls{SDCO} team.
Generation of \glspl{SRDP} is also the responsibility of \gls{SDCO}.
}
\label{fig:sdc-organization}
\end{figure}
\subsection{\acrlong{SDCO}}
\label{sec:governance:structure:sdco}
The \gls{SDC} is intended to be a long-term facility within ASTRON, broadly equivalent to a major piece of instrumentation like \gls{LOFAR}.
As such, it will be run by the \gls{SDCO} Operational Unit within ASTRON's \gls{AO} Department.
The overriding goal of the \gls{SDCO} team is to deliver effective services and data product to the community.
\gls{SDCO} is responsible for day-to-day facility operations.
This includes configuring, resourcing, and managing all services provided by the facility, as well as identifying and responding to problems.
Where those problems are software bugs, they may either be addressed directly, or, when necessary, referred to the \gls{SDCP}, as described in \cref{sec:governance:structure:sdcp}.
\gls{SDCO} will provide ``helpdesk''-type support to facility users.
This will include establishing and operating ticketing systems, discussion forums, mailing lists, etc, as appropriate.
\gls{SDCO} will provide training, tutorials, documentation, and other material to end users.
\gls{SDCO} is responsible for provisioning the infrastructure on which the \gls{SDC} facility will run.
This infrastructure may be directly purchased or leased, sourced from cloud or other vendors, or from in-kind contributions to the \gls{ILT} or other projects, with the agreement of the \gls{ILT} Board where appropriate.
Infrastructure will be consistent with the agreed-upon \gls{SDC} system architecture.
\gls{SDCO} is jointly responsible with the \gls{SDCP} for developing the overall architecture of the \gls{SDC} facility.
It is expected that this architecture will reflect current and likely future infrastructure available to the \gls{SDCO} team, as well as their operational requirements.
\gls{SDCO} will provide resources for maintenance of the software products delivered by \gls{SDCP} on which the \gls{SDC} facility relies, with the ultimate aim of ensuring that both the software itself and the associated developer expertise is sustainable beyond the duration of the \gls{SDCP} itself.
Even when funded by \gls{SDCO}, developers working on \gls{SDCP} products will follow the working practices, norms, policies, and procedures established by \gls{SDCP}.
\gls{SDCO} will work with the community and other stakeholders to identify both long- and short-term scientific and functional priorities for the \gls{SDC}.
These will be communicated to the \gls{SDCP}, and form the basis of a collaborative process of defining detailed requirements and prioritizing work.
\gls{SDCO} is responsible for acceptance testing and commissioning of the products delivered by \gls{SDCP}.
\gls{SDCO} is responsible for using the functionality provided by the \gls{SDC} facility to generate a set of standardized data products and to make them available to end users.
The \gls{SDCO} Operational Unit has some permanent staffing funded by the ASTRON base budget.
Additional members of the operations team will be drawn from Focus and Competence Groups as required.
The \gls{SDCO} Operational Unit Lead is responsible to the Head of \gls{AO}.
\subsection{\acrshort{SDC} Steering Committee}
\label{sec:governance:structure:steering}
The \gls{SDC} Steering Committee is a standing committee which was established by the ASTRON \gls{MT} to oversee all aspects of the \gls{SDC}.
In particular, it will provide a forum for resolving any differences or inconsistencies between the approaches taken by \gls{SDCO} and \gls{SDCP}.
\section{Community Engagement}
Beyond the formal management structure outlined above, we regard it as vital that the SDC be responsive to the current and emergent needs of the scientific community and other facilities.
By engaging with them directly, we can best ensure that the SDC is achieving its aims, and plan for and prioritise future work.
We therefore expect the SDC team to maintain regular contact with:
\begin{itemize}
\item the Board of the \gls{ILT};
\item the \gls{LUC};
\item the various \gls{LOFAR}\glspl{KSP};
\item other relevant organizations representing sections of the scientific community;
\item the \gls{SRC} network.
\end{itemize}
Engagement with \gls{ILT} Board is realised through a regular dialogue every trimester.
The \gls{SDCO} team provides updates about the activities relevant to \gls{LOFAR} and receives feedback from the Board concerning the areas that should receive focus in the future.
The \gls{SDCO} also has a regular dialogue with the \gls{LUC} every quarter, during which they offer a comprehensive overview of the achieved results, active development areas, and planning.
Following this, the \gls{LUC} will summarize the feedback from its members, as well as that the \glspl{KSP} it represents, and share it with \gls{SDCO}.
Dialogue with the \gls{SRC} network is realised through the engagement and contribution of various \gls{SDC} representatives to the \gls{SRCSC} working groups.
In addition, we expect to hold a programme of \gls{SDC} schools, traineeships, workshops, and other educational events which will serve not only to inform the community about the capabilities available through the \gls{SDC} but also to provide fora through which the \gls{SDC} team can solicit feedback directly from community members.
\section{Access Policy and Data Rights}
We envision the SDC as being accessible to all comers, with no restrictions on who is able to sign up for an account and log in to the basic service.
However:
\begin{itemize}
\item Policies will be defined for the use of resources such as storage, computing, and data transfer: simply having the ability to access the \gls{SDC} will not guarantee users with substantial capacity.
\item Users will have to submit proposals for substantial requests or, when requesting observing time, for the generation of advanced data products through the standard ASTRON pipelines.
If awarded, these requests will be handled by \gls{SDCO}\footnote{Expert users may be asked to participate in this following the current model of ‘user shared-support’ mode.}, who will deliver the final data products to the users.
For processing requests within a certain quota (to be defined), users will be able to directly use the \gls{SDC} resources and will not need to go through the proposal submission process.
\item The policy detailing the resource allocation will need further definition beyond its current concept state.
However, we note here simply that \gls{SDC} systems must be able to effectively implement this policy.
\item Data stored within and made available through the \gls{SDC} will be subject to ownership and data rights policies which will vary based on its origin.
Again, \gls{SDC} systems must be capable of recognizing and enforcing these policies.
\end{itemize}
Given an increasing move towards a model of cloud-based compute and storage, members of the user community may be able to provision their own compute and/or storage capacity on systems co-located with the SDC.
As such, users could directly access SDC resources at their own cost.
We suggest that this might be a worthwhile avenue for future development, but is not a priority for the SDC at the present time.
