Warn when a process that has imported tiledb calls fork() (#1876)

TileDB uses multiple threads and it's easy to run into deadlocks on Linux where `fork` is the default for `multiprocessing` until Python 3.14. We want to see a warning like Python 3.12 emits when fork() is called with multiple threads detected (see the implementation in posixmodule.c). I see three different ways to do this.

1. Do nothing other than document the situation and wait for users to migrate to Python 3.12.
2. Warn about fork() immediately on import of tiledb.
3. Warn when `os.fork()` is called and tiledb has been imported.  

The pros for number one: only needing to reference the change coming in 3.12. The cons: this project currently supports 4 versions of Python before 3.12 and it'll be years before new users are entirely on 3.12. Also, to a first approximation, nobody reads documentation. Perhaps better said as: we often try to read as little documentation as we can get away with.

The pros for number two: super easy to implement and all tiledb users see it. The cons: users would see the warning even if they had no intention of using `fork()`. It's noisy and would have to be actively silenced.

The pros for number three: all tiledb users who call `os.fork()`, intentionally or not, get the warning and no one else. It's not noisy. The cons: it requires monkeypatching`os.fork()` or, suboptimally, using `os.register_at_fork()`.

The only problem with `os.register_at_fork()` is that exceptions and warnings can't be raised from registered callables. We're limited to printing to stderr/stdout. This is a less good experience for developers and the project also currently checks in tests that tiledb doesn't print such output.

Monkeypatching methods of the standard library isn't a perfect solution. We'd all prefer that modules we import incidentally didn't change things in the standard library, right? And there's no guarantee that a module imported after tiledb won't patch `os.fork()` again.

On the other hand, users import tiledb deliberately, to use it. And it's mostly incompatible with `fork()`. The monkeypatch I'm suggesting for `os.fork()` doesn't change behavior of the stdlib function, it only wraps it in a warning. It's as innocuous as such a patch can be.

--- Original commit comments ---

* Add a failing test

We want to get a DeprecationWarning like Python 3.12 emits when
fork() is called with multiple threads detected.

* Monkeypatch os.fork and raise a warning

This is equivalent to the warnings added in Python 3.12. We don't
try to count threads because TileDB is multithreaded.

* Isolate built-in os.fork from in-test patching

* Don't monkeypatch on Windows

* Add a module level flag indicating whether os.fork needs a patch

Drop use of monkeypatch fixture for os.fork isolation. Add some
docs and comments to the two new fixtures and rename the test
module.

* Never access os.fork on windows

* Make isolate_os_fork fixture yield on all platforms

* Add missing win32 skip marks for two tests

* Add another comment about patching and a history entry

Author: sgillies

HISTORY.md

@@ -1,3 +1,9 @@
+# Next
+
+## Improvements
+
+* Warn when `os.fork()` is used in the presence of a Tiledb context [#1876](https://github.com/TileDB-Inc/TileDB-Py/pull/1876/files).
+
 # Release 0.24.0
 
 * TileDB-Py 0.24.0 includes TileDB Embedded [2.18.2](https://github.com/TileDB-Inc/TileDB/releases/tag/2.18.2)

tiledb/ctx.py

@@ -10,6 +10,7 @@
 _ctx_var = ContextVar("ctx")
 
 already_warned = False
+_needs_fork_wrapper = sys.platform != "win32" and sys.version_info < (3, 12)
 
 
 class Config(lt.Config):
@@ -345,6 +346,16 @@ def __init__(self, config: Config = None):
 
         self._set_default_tags()
 
+        # The core tiledb library uses threads and it's easy
+        # to experience deadlocks when forking a process that is using
+        # tiledb.  The project doesn't have a solution for this at the
+        # moment other than to avoid using fork(), which is the same
+        # recommendation that Python makes. Python 3.12 warns if you
+        # fork() when multiple threads are detected and Python 3.14 will
+        # make it so you never accidentally fork(): multiprocessing will
+        # default to "spawn" on Linux.
+        _ensure_os_fork_wrap()
+
     def __repr__(self):
         return "tiledb.Ctx() [see Ctx.config() for configuration]"
 
@@ -439,7 +450,6 @@ def check_ipykernel_warn_once():
     global already_warned
     if not already_warned:
         try:
-            import sys
             import warnings
 
             if "ipykernel" in sys.modules and tuple(
@@ -521,7 +531,46 @@ def default_ctx(config: Union["Config", dict] = None) -> "Ctx":
         ctx = _ctx_var.get()
         if config is not None:
             raise tiledb.TileDBError("Global context already initialized!")
+
+        # The core tiledb library uses threads and it's easy
+        # to experience deadlocks when forking a process that is using
+        # tiledb.  The project doesn't have a solution for this at the
+        # moment other than to avoid using fork(), which is the same
+        # recommendation that Python makes. Python 3.12 warns if you
+        # fork() when multiple threads are detected and Python 3.14 will
+        # make it so you never accidentally fork(): multiprocessing will
+        # default to "spawn" on Linux.
+        _ensure_os_fork_wrap()
     except LookupError:
         ctx = tiledb.Ctx(config)
         _ctx_var.set(ctx)
     return ctx
+
+
+def _ensure_os_fork_wrap():
+    global _needs_fork_wrapper
+    if _needs_fork_wrapper:
+        import os
+        import warnings
+        from functools import wraps
+
+        def warning_wrapper(func):
+            @wraps(func)
+            def wrapper():
+                warnings.warn(
+                    "TileDB is a multithreading library and deadlocks "
+                    "are likely if fork() is called after a TileDB "
+                    "context has been created (such as for array "
+                    "access). To safely use TileDB with "
+                    "multiprocessing or concurrent.futures, choose "
+                    "'spawn' as the start method for child processes. "
+                    "For example: "
+                    "multiprocessing.set_start_method('spawn').",
+                    UserWarning,
+                )
+                return func()
+
+            return wrapper
+
+        os.fork = warning_wrapper(os.fork)
+        _needs_fork_wrapper = False

tiledb/tests/conftest.py

@@ -1,4 +1,5 @@
 import ctypes
+import os
 import sys
 
 import pytest
@@ -98,3 +99,24 @@ def __init__(self, params=None):
 
     tiledb.Ctx = PatchedCtx
     tiledb.Config = PatchedConfig
+
+
+@pytest.fixture(scope="function", autouse=True)
+def isolate_os_fork(original_os_fork):
+    """Guarantee that tests start and finish with no os.fork patch."""
+    # Python 3.12 warns about fork() and threads. Tiledb only patches
+    # os.fork for Pythons 3.8-3.11.
+    if original_os_fork:
+        tiledb.ctx._needs_fork_wrapper = True
+        os.fork = original_os_fork
+    yield
+    if original_os_fork:
+        tiledb.ctx._needs_fork_wrapper = True
+        os.fork = original_os_fork
+
+
+@pytest.fixture(scope="session")
+def original_os_fork():
+    """Provides the original unpatched os.fork."""
+    if sys.platform != "win32":
+        return os.fork

tiledb/tests/test_fork_ctx.py

@@ -0,0 +1,102 @@
+"""Tests combining fork with tiledb context threads.
+
+Background: the core tiledb library uses threads and it's easy to
+experience deadlocks when forking a process that is using tiledb.  The
+project doesn't have a solution for this at the moment other than to
+avoid using fork(), which is the same recommendation that Python makes.
+Python 3.12 warns if you fork() when multiple threads are detected and
+Python 3.14 will make it so you never accidentally fork():
+multiprocessing will default to "spawn" on Linux.
+"""
+
+import multiprocessing
+import os
+import sys
+import warnings
+
+import pytest
+
+import tiledb
+
+
+@pytest.mark.skipif(
+    sys.platform == "win32", reason="fork() is not available on Windows"
+)
+def test_no_warning_fork_without_ctx():
+    """Get no warning if no tiledb context exists."""
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        pid = os.fork()
+        if pid == 0:
+            os._exit(0)
+        else:
+            os.wait()
+
+
+@pytest.mark.skipif(
+    sys.platform == "win32", reason="fork() is not available on Windows"
+)
+def test_warning_fork_with_ctx():
+    """Get a warning if we fork after creating a tiledb context."""
+    _ = tiledb.Ctx()
+    with pytest.warns(UserWarning, match="TileDB is a multithreading library"):
+        pid = os.fork()
+        if pid == 0:
+            os._exit(0)
+        else:
+            os.wait()
+
+
+@pytest.mark.skipif(
+    sys.platform == "win32", reason="fork() is not available on Windows"
+)
+def test_warning_fork_with_default_ctx():
+    """Get a warning if we fork after creating a default context."""
+    _ = tiledb.default_ctx()
+    with pytest.warns(UserWarning, match="TileDB is a multithreading library"):
+        pid = os.fork()
+        if pid == 0:
+            os._exit(0)
+        else:
+            os.wait()
+
+    pass
+
+
+@pytest.mark.skipif(
+    sys.platform == "win32", reason="fork() is not available on Windows"
+)
+def test_no_warning_multiprocessing_without_ctx():
+    """Get no warning if no tiledb context exists."""
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        mp = multiprocessing.get_context("fork")
+        p = mp.Process()
+        p.start()
+        p.join()
+
+
+@pytest.mark.skipif(
+    sys.platform == "win32", reason="fork() is not available on Windows"
+)
+def test_warning_multiprocessing_with_ctx():
+    """Get a warning if we fork after creating a tiledb context."""
+    _ = tiledb.Ctx()
+    mp = multiprocessing.get_context("fork")
+    p = mp.Process()
+    with pytest.warns(UserWarning, match="TileDB is a multithreading library"):
+        p.start()
+    p.join()
+
+
+@pytest.mark.skipif(
+    sys.platform == "win32", reason="fork() is not available on Windows"
+)
+def test_warning_multiprocessing_with_default_ctx():
+    """Get a warning if we fork after creating a default context."""
+    _ = tiledb.default_ctx()
+    mp = multiprocessing.get_context("fork")
+    p = mp.Process()
+    with pytest.warns(UserWarning, match="TileDB is a multithreading library"):
+        p.start()
+    p.join()