Skip to content

RAP-258 Offload writing to a separate thread

Mark de Wever requested to merge RAP-258-use-write-thread into master

When the processing writes to a new MS the writing of a buffer can be done while a new buffer is processed. This has some overhead by creating a deep copy of the buffer. However this copy is faster than the writing so it is a win for the total processing time. For total CPU usage this has a cost overhead.

A test on DAS6 with the parset as used in the ticket:

  • Using 500 baselines of a MS
  • split in 10 steps
    • phase shift
    • average
    • write

Results:

  • Before: Total DP3 time 237.86 real 500.92 user 228.21 system
  • After : Total DP3 time 211.63 real 519.11 user 286.01 system

The same test with 20000 baselines of a MS as used on real data gives

  • Before: Total DP3 time 3551.98 real 3724.33 user 16390.3 system
  • After : Total DP3 time 3019.5 real 3759.75 user 16238.7 system

Closes RAP-258

Edited by Mark de Wever

Merge request reports