Skip to content

Speed-up attenuate/apply_beam function

Marcel Loose requested to merge RAP-690_Speed-up-skymodel-creation into master

Analysis

Analysis with py-spy of a number of the many sub-processes started when building up the sky model revealed that a significant amount of time is spent in the call eb.load_telescope(beamMS)). This function is called for every source, which is very inefficient, especially when the MS files reside on a slow (shared) disk. Limiting the number of function calls can significantly reduce the run time of this step.

Since the functions attenuate() and apply_beam() were written, EveryBeam was improved such that the array_factor() function can now handle arrays of directions. So there's no longer a need to parallelize over source directions in LSMTool.

Implementation

Rewrote the attenuate function, and renamed it to apply_beam. The new apply_beam function now does a single call to eb.load_telescope(beamMS), instead of many thousands. Basically all the logic in the old attenuate and apply_beam functions have been combined in the new apply_beam function.

On my workstation, where I/O is not the bottle-neck, run-times where reduced by roughly a factor of 30. On systems with a slow shared disk system, even more speed-up is expected.

Closes RAP-690

Edited by Marcel Loose

Merge request reports