Skip to content
Snippets Groups Projects

Speed-up attenuate/apply_beam function

Merged Marcel Loose requested to merge RAP-690_Speed-up-skymodel-creation into master
All threads resolved!

Analysis

Analysis with py-spy of a number of the many sub-processes started when building up the sky model revealed that a significant amount of time is spent in the call eb.load_telescope(beamMS)). This function is called for every source, which is very inefficient, especially when the MS files reside on a slow (shared) disk. Limiting the number of function calls can significantly reduce the run time of this step.

Since the functions attenuate() and apply_beam() were written, EveryBeam was improved such that the array_factor() function can now handle arrays of directions. So there's no longer a need to parallelize over source directions in LSMTool.

Implementation

Rewrote the attenuate function, and renamed it to apply_beam. The new apply_beam function now does a single call to eb.load_telescope(beamMS), instead of many thousands. Basically all the logic in the old attenuate and apply_beam functions have been combined in the new apply_beam function.

On my workstation, where I/O is not the bottle-neck, run-times where reduced by roughly a factor of 30. On systems with a slow shared disk system, even more speed-up is expected.

Closes RAP-690

Edited by Marcel Loose

Merge request reports

Merge request pipeline #87490 passed

Merge request pipeline passed for 127946b1

Merged by Marcel LooseMarcel Loose 7 months ago (Jul 9, 2024 12:50pm UTC)

Loading

Pipeline #87581 passed

Pipeline passed for ae225238 on master

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Looks good! Just one minor question.

  • David Rafferty approved this merge request

    approved this merge request

  • merged

  • Marcel Loose mentioned in commit ae225238

    mentioned in commit ae225238

  • Marcel Loose resolved all threads

    resolved all threads

  • Please register or sign in to reply
    Loading