Fix Python 3.14 stack overflow in NonlinearSolveSciPy tests #732
+44
−18
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes CI failures caused by Python 3.14.0's buggy stack overflow detection. The error "Fatal Python error: _Py_CheckRecursiveCall: Unrecoverable stack overflow" with negative memory values indicates a calculation bug in Python 3.14's new stack detection mechanism when interacting with Julia's task system. This issue affects: - Julia 1.9, 1.12 - macOS and Linux - Python 3.14.0 Pinning Python to >= 3.9, < 3.14 resolves the issue. References: - PythonCall.jl issue: JuliaPy/PythonCall.jl#694 - Python upstream issues: python/cpython#139653, python/cpython#137573 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Fixes test failures where Julia-specific keyword arguments (alias, verbose) were being forwarded to scipy.optimize functions that don't recognize them. The error was: ``` Python: TypeError: least_squares() got an unexpected keyword argument 'alias' Python: TypeError: root() got an unexpected keyword argument 'alias' ``` Now filters out :alias and :verbose kwargs in all three __solve methods: - SciPyLeastSquares - SciPyRoot - SciPyRootScalar 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Fixes two critical issues in Python interop:
1. **Type conversion errors**: Replace direct Vector{Float64}() and Float64()
constructors with pyconvert() for proper PythonCall type conversion.
- Import pyconvert from PythonCall
- Update _make_py_residual to use pyconvert(Vector{Float64}, x_py)
- Update _make_py_scalar to use pyconvert(Float64, x_py)
- Update all result conversions (res.x, res.fun) to use pyconvert
Fixes error:
```
MethodError: no method matching Vector{Float64}(::PythonCall.Py)
```
2. **Kwargs splatting errors**: Change from filter() iterator to pairs generator
for proper kwargs splatting to Python functions.
- Replace scipy_kwargs = filter(...) with inline pairs generator
- Use (k => v for (k, v) in pairs(kwargs) if k ∉ (:alias, :verbose))...
Fixes error:
```
TypeError: object of type 'NoneType' has no len()
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
Fixes two critical issues discovered in CI: 1. **Kwargs splatting error**: Convert generator to Tuple before splatting - Change from inline generator: `(k => v for ...)...` - To collected Tuple: `scipy_kwargs = Tuple(k => v for ...); scipy_kwargs...` - Fixes: `TypeError: object of type 'NoneType' has no len()` 2. **Boolean context errors**: Use pyconvert for Python boolean fields - `res.success` → `pyconvert(Bool, res.success)` - `res.converged` → `pyconvert(Bool, res.converged)` - Fixes: `TypeError: non-boolean (PythonCall.Py) used in boolean context` Applied to all three solver methods: - SciPyLeastSquares (line 137, 151) - SciPyRoot (line 177, 190) - SciPyRootScalar (line 218, 230) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Fixes error: `TypeError: object of type 'NoneType' has no len()` Problem: We were passing `bounds=None` to scipy when bounds weren't specified, but scipy.optimize.least_squares tries to call len() on the bounds parameter, which fails on None. Solution: Conditionally omit the bounds kwarg entirely when not specified, rather than passing Python's None. This allows scipy to use its default bounds of (-inf, inf). Split the scipy call into two branches: - Without bounds: When prob doesn't have lb/ub constraints - With bounds: When prob has lb/ub constraints 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes CI failures in NonlinearSolveSciPy tests on Julia v1.12 caused by Python 3.14.0's buggy stack overflow detection mechanism.
Problem
The CI is failing with:
Note the negative memory values - this indicates a calculation bug in Python 3.14's new stack overflow detection, not an actual stack overflow.
Root Cause
Python 3.14 introduced a redesigned stack overflow detection mechanism that has multiple known bugs:
This is not a PythonCall.jl or NonlinearSolveSciPy issue - it's an upstream Python 3.14 bug affecting many projects.
Solution
Added
CondaPkg.tomlto pin Python to>=3.9,<3.14, ensuring Python 3.13.x is installed instead.References
Test Plan
CI should pass with Python 3.13.x instead of 3.14.0.
🤖 Generated with Claude Code