std exception detected when applying solutions in LINC target workflow
Hi,
when running the LINC target workflow I get a crash with DP3 at the prep step.
DP3 version is:
Apptainer> DP3 --version DP3 6.0.0 6.0.1-66-g66525837 This is currently adopted for packaging of LINC in docker-hub TAG linc-v4.0-83-g0fd8896
The error ir reported below: Starting program: /usr/local/bin/DP3 /project/lspc/Data/2MASS/TMPxqvvntt4/stg1fb8fc13-de37-492e-ba78-2c5a5c325527/DP3.parset applyPA.parmdb=/project/lspc/Data/2MASS/TMPxqvvntt4/stgb09eb0c2-ae0b-47cc-8fcf-0f027112e6d3/cal_solutions.h5 applybandpass.parmdb=/project/lspc/Data/2MASS/TMPxqvvntt4/stgb09eb0c2-ae0b-47cc-8fcf-0f027112e6d3/cal_solutions.h5 applyclock.parmdb=/project/lspc/Data/2MASS/TMPxqvvntt4/stgb09eb0c2-ae0b-47cc-8fcf-0f027112e6d3/cal_solutions.h5 applyphase.parmdb=/project/lspc/Data/2MASS/TMPxqvvntt4/stgb09eb0c2-ae0b-47cc-8fcf-0f027112e6d3/cal_solutions.h5 applyRM.parmdb=/project/lspc/Data/2MASS/TMPxqvvntt4/stgb09eb0c2-ae0b-47cc-8fcf-0f027112e6d3/cal_solutions.h5 applytec.parmdb=/project/lspc/Data/2MASS/TMPxqvvntt4/stgb09eb0c2-ae0b-47cc-8fcf-0f027112e6d3/cal_solutions.h5 msin.baseline=[CR]S*& msout.storagemanager.databitrate=0 msin.datacolumn=DATA count1.jsonfilename=out1.json count2.jsonfilename=out2.json numthreads=10 msin=/project/lspc/Data/2MASS/TMPxqvvntt4/stg70c7359c-4f6d-411a-8811-b958e8a23a5d/L694223_SB000_uv.MS msout=L694223_SB000_uv.MS msout.datacolumn=DATA count1.savetojson=True count2.savetojson=True demix.skymodel=/project/lspc/Data/2MASS/TMPxqvvntt4/stgf4df8e48-01c7-47d4-aeae-90a6c3a541a7/output.skymodel msout.storagemanager=Dysco [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7ffff04a2640 (LWP 1246699)] [New Thread 0x7fffefca1640 (LWP 1246700)] [New Thread 0x7fffef4a0640 (LWP 1246701)] [New Thread 0x7fffeec9f640 (LWP 1246702)] [New Thread 0x7fffee49e640 (LWP 1246704)] [New Thread 0x7fffedc9d640 (LWP 1246705)] [New Thread 0x7fffed49c640 (LWP 1246706)] [New Thread 0x7fffecc9b640 (LWP 1246707)] [New Thread 0x7fffec49a640 (LWP 1246708)] [New Thread 0x7fffeba94640 (LWP 1246783)] [New Thread 0x7fffeb0d7640 (LWP 1246784)] [New Thread 0x7fffea71a640 (LWP 1246785)] [New Thread 0x7fffe9d5d640 (LWP 1246786)] [New Thread 0x7fffe93a0640 (LWP 1246787)] [New Thread 0x7fffe89e3640 (LWP 1246788)] [New Thread 0x7fffd3fff640 (LWP 1246789)] [New Thread 0x7fffd37fe640 (LWP 1246790)] [New Thread 0x7fffd2e16640 (LWP 1246791)] [New Thread 0x7fffd2459640 (LWP 1246792)] [New Thread 0x7fffd1c2d640 (LWP 1246793)] [New Thread 0x7fffd1401640 (LWP 1246794)] [New Thread 0x7fffd0bd5640 (LWP 1246795)] [New Thread 0x7fffbbfff640 (LWP 1246796)] [New Thread 0x7fffbb7fe640 (LWP 1246797)] [New Thread 0x7fffbaffd640 (LWP 1246798)] MSReader selecting baselines ...
Thread 1 "DP3" hit Catchpoint 1 (exception thrown), 0x00007ffff73f14a1 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6 (gdb) bd Undefined command: "bd". Try "help". (gdb) bt #0 0x00007ffff73f14a1 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6 #1 (closed) 0x00007ffff7649f52 in ?? () from /lib/x86_64-linux-gnu/libcasa_casa.so.6 #2 (closed) 0x00007ffff76a6b3b in casacore::Array<bool, std::allocator >::operator()(casacore::IPosition const&, casacore::IPosition const&, casacore::IPosition const&) () from /lib/x86_64-linux-gnu/libcasa_casa.so.6 #3 (closed) 0x00007ffff76aaa6c in casacore::Array<bool, std::allocator >::operator()(casacore::IPosition const&, casacore::IPosition const&) () from /lib/x86_64-linux-gnu/libcasa_casa.so.6 #4 0x00007ffff7c45934 in dp3::base::BaselineSelection::handleBL(casacore::Matrix<bool, std::allocator >&, dp3::base::DPInfo const&) const () from /usr/local/lib/libDP3.so #5 0x00007ffff7c467d9 in dp3::base::BaselineSelection::apply(dp3::base::DPInfo const&) const () from /usr/local/lib/libDP3.so #6 (closed) 0x00007ffff7d09fe0 in dp3::steps::Filter::updateInfo(dp3::base::DPInfo const&) () from /usr/local/lib/libDP3.so #7 0x00007ffff7e1319b in dp3::steps::Step::setInfo(dp3::base::DPInfo const&) () from /usr/local/lib/libDP3.so #8 (closed) 0x00007ffff7cf610d in dp3::steps::Demixer::updateInfo(dp3::base::DPInfo const&) () from /usr/local/lib/libDP3.so #9 (closed) 0x00007ffff7e1319b in dp3::steps::Step::setInfo(dp3::base::DPInfo const&) () from /usr/local/lib/libDP3.so #10 (closed) 0x00007ffff7c5fddb in dp3::base::Execute(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, char**) () from /usr/local/lib/libDP3.so #11 (closed) 0x0000555555556771 in main () (gdb) continue Continuing. [Thread 0x7fffd37fe640 (LWP 1246790) exited] [Thread 0x7fffe89e3640 (LWP 1246788) exited] [Thread 0x7fffe93a0640 (LWP 1246787) exited] [Thread 0x7fffe9d5d640 (LWP 1246786) exited] [Thread 0x7fffd3fff640 (LWP 1246789) exited] [Thread 0x7fffea71a640 (LWP 1246785) exited] [Thread 0x7fffeb0d7640 (LWP 1246784) exited] [Thread 0x7fffeba94640 (LWP 1246783) exited] [Thread 0x7fffd1c2d640 (LWP 1246793) exited] [Thread 0x7fffd2e16640 (LWP 1246791) exited] [Thread 0x7fffd0bd5640 (LWP 1246795) exited] [Thread 0x7fffbaffd640 (LWP 1246798) exited] [Thread 0x7fffd2459640 (LWP 1246792) exited] [Thread 0x7fffbbfff640 (LWP 1246796) exited] [Thread 0x7fffd1401640 (LWP 1246794) exited] [Thread 0x7fffbb7fe640 (LWP 1246797) exited]
**std exception detected: ArrayBase::operator()(b,e,i) - incorrectly specified begin: [0, 0] end: [72, 72] incr: [1, 1]
array shape: [60, 60] required: b >= 0; b <= e; e < shape; i >= 0**
[Thread 0x7fffecc9b640 (LWP 1246707) exited] [Thread 0x7fffec49a640 (LWP 1246708) exited] [Thread 0x7fffed49c640 (LWP 1246706) exited] [Thread 0x7fffedc9d640 (LWP 1246705) exited] [Thread 0x7fffee49e640 (LWP 1246704) exited] [Thread 0x7fffeec9f640 (LWP 1246702) exited] [Thread 0x7fffef4a0640 (LWP 1246701) exited] [Thread 0x7fffefca1640 (LWP 1246700) exited] [Thread 0x7ffff04a2640 (LWP 1246699) exited] [Inferior 1 (process 1244857) exited with code 01] (gdb)
The DP3.parset is:
steps = [filter,count1,flagbaseline,flagelev,flagamp,demix,applyPA,applybandpass,applyclock,applybeam,applyRM,avg,count2]
msout.storagemanager = "Dysco" msout.storagemanager.databitrate = 0
count1.type = counter count2.type = counter
filter.type = filter filter.baseline = [CR]S*& filter.remove = true
flagedge.type = preflagger flagedge.chan = [0..nchan/32-1,31*nchan/32..nchan-1]
aoflag.type = aoflagger aoflag.memoryperc = 20 aoflag.keepstatistics = false
flagbaseline.type = preflagger flagbaseline.baseline = []
flagelev.type = preflagger flagelev.elevation = 0deg..15deg
flagamp.type = preflagger flagamp.amplmin = 1e-30
applyPA.type = applycal applyPA.correction = polalign applyPA.solset = calibrator
applybandpass.type = applycal applybandpass.correction = bandpass applybandpass.updateweights = True applybandpass.solset = calibrator
applyclock.type = applycal applyclock.correction = clock applyclock.solset = calibrator
applytec.type = applycal applytec.correction = tec applytec.solset = calibrator
applyphase.type = applycal applyphase.correction = phaseOrig applyphase.solset = calibrator
applyRM.type = applycal applyRM.correction = RMextract applyRM.solset = target
applybeam.type = applybeam applybeam.usechannelfreq = True applybeam.updateweights = True applybeam.invert = True applybeam.beammode = element
avg.type = average avg.timeresolution = 4 avg.freqresolution = 48.82kHz
demix.type = demixer demix.baseline = [CR]S*& demix.demixfreqresolution = 48.82kHz demix.demixtimeresolution = 10 demix.ignoretarget = False demix.targetsource = 2MASX demix.subtractsources = ["CasA_4_patch"] demix.ntimechunk = 10 demix.freqstep = 1 demix.timestep = 1 demix.instrumentmodel = instrument demix.uselbfgssolver = True demix.lbfgs.historysize = 10 demix.lbfgs.robustdof = 200 demix.maxiter = 20