Skip to content

Prepare GainCal step for aocommon's threading system

This prepares GainCal for the updates to the threading system in aocommon.

A few particulars:

  • The system used to be that the frequency channels were parallelized by a part of the threads, and the rest of the threads would be used to parallelize over stations in the unpolarized GainCal algorithm. It's not clear why this isn't done for the polarized algorithm -- @dijkema any idea?
  • This used to be done with separate DynamicFor loops, but with the new thread pool class that will not work, because there would be a loop inside a loop, whereas the outer loop would already have claimed all threads. Therefore, this is now a RecursiveFor.
  • There are a few parallelized loops, e.g. over rather simple initialization that I doubt will speed up processing. The first loop inside GainCalAlgorithm::doStep_unpolarized is an example. It probably won't hurt much either, but it might be worthwhile in the future to time such things to be sure.
  • In the new aocommon system, constructing the threading "For" classes is relatively cheap, so they can be constructed when they are used, instead of storing them at class scope. This is particularly important for the Predict class, which can store a ThreadPool to allow for recursive threading. This system was used here in GainCal, but not to enable recursive threading, but just to keep the threads alive, which will no longer be necessary.

A part of this MR may improve things because of better scheduling of recursive work by the RecursiveFor class. However, with the old aocommon version, threads will be restarted, so this will temporarily have a negative effect until we update aocommon.

Merge request reports

Loading