Skip to content

AST-1149 Properly handle GIL in PythonDeconvolution class

The function PythonDeconvolution::ExecuteMajorIteration is used multi-threaded. The calls to python functions through pybind11 were protected by a mutex. This indeed prevented concurrent calls to non-thread-safe python code.

However, the proper way to protect python calls is to obtain the Global Interpreter Lock (GIL). There are some extra checks in pybind11 to ensure the GIL is held when non-thread-safe code is called. These checks are not included in production code (CMAKE_BUILD_TYPE=Release). The CI of wsclean is build without optimizations (CMAKE_BUILD_TYPE="") and triggers the pybind11 GIL checks when this test is run:

cd <wsclean_build_dir>/tests/python
pytest source/

The solution is to obtain the GIL instead of using a mutex to protect python code. For threads to be able to obtain the GIL, the main thread needs to release it first. This happens at the end of the constructor of PythonDeconvolution.

Edited by Bas van der Tol

Merge request reports