Sarod Yatawatta (ecf5311e) at 22 Feb 09:44
upgrade to ubuntu22.04
After the CPU only version is compiled and installed, it reports errors when importing idg, for example
>>> import idg Error: No module named 'idg.CUDA' Error: No module named 'idg.HybridCUDA'
this should only be a warning
Yes, I have copied data locally.
Yes, I need to copy the files to reproduce this
Could you let it run to see what the output is?
You can do the test in the branch or the master, either is fine
Sarod Yatawatta (8096210c) at 09 Feb 08:24
fix linting
Sarod Yatawatta (b5ab29e8) at 09 Feb 08:14
Revert "Merge branch 'RAP-299-DDECal' into 'master'"
... and 4 more commits
Sarod Yatawatta (fe2606d5) at 08 Feb 08:47
Sarod Yatawatta (24c71a31) at 08 Feb 08:47
Merge branch 'sagecalpred_h5parm_singleton' into 'master'
... and 1 more commit
libhdf5 is not always thread safe, and it is not necessary to re-read the same h5 file when multiple threads are accessing it, this fixes both issues by using a singleton for the h5 file, the h5parm is passed as a reference to all threads (so h5parm is assumed thread safe, for reading)
libhdf5 is not always thread safe, and it is not necessary to re-read the same h5 file when multiple threads are accessing it, this fixes both issues by using a singleton for the h5 file, the h5parm is passed as a reference to all threads (so h5parm is assumed thread safe, for reading)
Its OK, you only need to build DP3
Reason: h5 library is not generally thread safe, this fixes this problem, but h5parm is assumed thread safe
Hi @alexKurek , I have created a DP3 branch https://git.astron.nl/RD/DP3/-/commits/sagecalpred_h5parm_singleton
which will fix the issue, please try and let me know.
Sarod Yatawatta (8f7d7227) at 31 Jan 10:56
add h5parm singleton
Ok thanks, what is the size of the h5parm file, like this /storage/akurek/largerTmp/toil-cwl-runner.r0tzhgin/stg54d294d6-c703-43fb-abfc-a9e1721e77ed/fast_phases.h5parm It seams to be reading multiple copies of this, which I need to fix.
Can you also send me the sky model to use when this is happening?