Skip to content

Fix: ProcessPoolExecutor uses fork instead of spawn on macOS #168

@stephenfeather

Description

@stephenfeather

Expected Behavior

ProcessPoolExecutor in math scripts should safely create worker processes without risk of crashes on macOS, regardless of what libraries have been imported in the parent process.

Actual Behavior

Both math_base.py (safe_compute) and sympy_compute.py (safe_solve) use ProcessPoolExecutor with the default start method, which is fork on macOS. Forking after importing libraries like numpy, sqlite3, or the macOS Accelerate framework can cause:

  • Segfaults / "Python quit unexpectedly" crashes
  • Deadlocks from forked mutex state
  • Corrupted state in child processes

This is a known macOS issue — Apple's Accelerate framework and other system libraries are not fork-safe.

Files Affected

  • scripts/cc_math/math_base.pysafe_compute() function
  • scripts/cc_math/sympy_compute.pysafe_solve() function

Fix Implemented

math_base.py

  1. Added import multiprocessing
  2. Use multiprocessing.get_context("spawn") and pass as mp_context to ProcessPoolExecutor
  3. Removed local _wrapper closure (not picklable with spawn) — submit func directly with *args, **kwargs
# Before
with ProcessPoolExecutor(max_workers=1) as executor:
    future = executor.submit(_wrapper)

# After
ctx = multiprocessing.get_context("spawn")
with ProcessPoolExecutor(max_workers=1, mp_context=ctx) as executor:
    future = executor.submit(func, *args, **kwargs)

sympy_compute.py

  1. Added import multiprocessing
  2. Use multiprocessing.get_context("spawn") and pass as mp_context to ProcessPoolExecutor
  3. No closure issue here — _solve_internal is already a module-level function
# Before
with ProcessPoolExecutor(max_workers=1) as executor:
    future = executor.submit(_solve_internal, equation, var, domain)

# After
ctx = multiprocessing.get_context("spawn")
with ProcessPoolExecutor(max_workers=1, mp_context=ctx) as executor:
    future = executor.submit(_solve_internal, equation, var, domain)

Testing

  • sympy_compute.py solve "x**2 - 4"[-2, 2]
  • safe_compute(math.factorial, 10)3628800
  • safe_solve("x**2 - 4")[-2, 2]

Note

The spawn start method is slightly slower than fork (it starts a fresh Python interpreter) but is the only safe option on macOS. It is already the default on Windows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions