Skip to content

Rationalize handling of IDA “workflows”

Historically, we've evinced various ambitious plans for advanced mapping between compute facilities and different types of workflow. These remain great ideas, and we should push for them in future development. However, as we approach the end of the project, we are left with an “interactive analysis” section of ESAP which has a number of flaws. Off the top of my head:

  • It lists a number of “workflows” which simply can't be executed by any available compute facilities because we haven't yet defined an execution mechanism (e.g. TOPCAT).
  • The “skip” button makes it possible to proceed without selecting a “workflow”, but this then fails when the user selects a BinderHub-based analysis facility (the BinderHub returns “400: Bad Request”).
  • Having selected a good workflow (e.g. a repository containing a notebook which can be used with BinderHub) the user is then offered the option to execute it on a service which doesn't support it (e.g. SKAO DLaaS).

...and there are are probably others.

In the short term — ie, for the conclusion of the project — we should aim to provide a simplified interface which does not provide all of the functionality we ultimately envision, but which is at least consistent and does not prompt users to select known-bad combinations of software and service providers.

Given that, I propose:

  • The list of available workflows should display only Jupyter notebook repositories which can be successfully executed through BinderHub based systems. This doesn't mean that we should delete other workflows, but simply that the IDA selection screen should filter on the jupyter-notebook keyword and not present the user with other options (until and unless we develop facility support for those other options).
  • The list of available facilities should only display BinderHub based systems which are capable of executing the workflows with the juptyer-notebook keyword. (Again, this doesn't mean we have to delete other facilities, but e.g. a “binderhub” keyword could be used to identify and only display appropriate facilities.)
  • In order to facilitate users who simply want a “blank” notebook to experiment with, we should add a “blank” workflow which just contains an empty notebook. This should be clearly described as such. The “skip” button can then be removed.

I think that covers all of the bases, but I'm happy to hear feedback on what I missed (particularly calling out @kliffen , @Voutsinas and @dmorris who I think might have views here, but everybody is welcome to comment).

My implementation suggestions above (paraphrased as “filter on keywords and add a blank repo”) are just suggestions, not requirements; I'm happy for the person implementing this to take another approach, but the functional result should be broadly as agreed on this ticket.