Deadlock when using fork in multiprocessing in Linux

[antonpetrov145]

Hello, I am experiencing an issue where I need to run these

with rawpy.imread(RAW_FILE) as raw:
        rgb = raw.postprocess()

in separate child process, but when I do that it deadlocks and I have to terminate the process forcefully

import rawpy
import multiprocessing
import time

RAW_FILE = "test.arw"  # put any RAW file here


def load_raw():
    print("reading")
    with rawpy.imread(RAW_FILE) as raw:
        rgb = raw.postprocess()
    print("finished")


def child_process():
    # Call inside child
    load_raw()
    # Second call
    load_raw()  # deadlock


if __name__ == "__main__":
    print("Start")
    load_raw()  # works

    # Start child process
    p = multiprocessing.Process(target=child_process)
    p.start()

    # Let it run for 10 seconds
    for _ in range(10):
        if not p.is_alive():
            break
        print("Child is still running")
        time.sleep(2)

    p.terminate()
    p.join()
    print("end")

When you run the above code, you will see this - not finishing ever in the forked process:

Start
reading
finished
Child is still running
reading
Child is still running
end

Which I read it like - when it goes to the child the function stops, so my question is - does this issue refer to libraw itself or can be fixed in rawpy?

[letmaik]

@antonpetrov145 Can you please review the info in the PR #279 and see if this solves it?

[antonpetrov145]

@letmaik thank you for the response, yes using spawn resolves the issue, but in my case is not possible to change to that mode, because my app has resource constraints, thus I have to use fork or move to separate threads and not using processes at all.

In any case it is a valid solution and this thread can be closed.

Thank you for the awesome library.