search function in ngtpy is not thread-safe

[y-oyama-tg]

I'd like to report a potential bug in ngtpy. I assumed the search function in ngtpy was thread-safe because it's a read-only operation, but I encountered the issue below. (search in C++ is thread-safe according to #117, but python binding seems to be not the case.)

Repro steps

Save the following code as main.py and execute python main.py in Python 3.10 or newer.

Expected result

the Python process should exit successfully after search.

Actual result

the Python process got frozen forever.

from concurrent.futures import ThreadPoolExecutor
import ngtpy
import random

dim = 10
query = [random.random() for _ in range(dim)]

ngtpy.create(b"tmp", dim)
index = ngtpy.Index(b"tmp")

# warm-up search. if you comment in this one line, the deadlock does not occur
# index.search(query, 1)


def f(i):
    # this causes a deadlock when initializing numpy inside ngtpy if the warm-up search is not run
    return index.search(query, 1)


with ThreadPoolExecutor() as executor:
    futs = []
    for i in range(2):
        fut = executor.submit(f, i)
        futs.append(fut)

    for fut in futs:
        print(fut.result())

Workaround

As described above, adding a warm-up search on the main thread is a workaround for this problem.

Potential cause

It looks like some lines like this triggers lazy import of numpy, which may not be thread-safe.