Skip to content

Read sky model only once

Andre Offringa requested to merge read-model-only-once into master

The Predict step of every direction reads its own copy of the sky model from disk. This reading & parsing takes 5-10 seconds for a 10K model. With 50 directions, this therefore takes a couple of minutes.

While a couple of minutes is mostly peanuts when processing big data sets, it does making debugging slower. So, while it's always nice to shave off a few minutes of processing, the main reason to fix this is to make debugging faster.

This MR introduces a sky model cache that stores the 'unfiltered' model (i.e. before selecting the direction). I've tested it on my run and it seems to i) still read the model correctly; and ii) speeds initialization up from minutes to seconds.

I've also renamed one of the two SourceDB classes to SourceDBWrapper, because it's a bad idea to have two classes with the same name. A lot more refactoring can be done there -- the way that the SourceDBWrapper is implemented by previous work is far more complex than necessary, and the naming of things is not logical. I leave that for another MR.

We should think about removing support for the SourceDB format. The code is complex and not in line with current style.

Merge request reports