fix compatible issues in python 3.14 #236

cxzhong · 2025-10-02T14:23:24Z

This PR fixes the compatible problem with python 3.14.

Do not call python code in the signal handler. I just use the async signal safe functions. I just add signal_to_raise record the signal and call python exception in the safety context. (fix it with more straight way)
I fixed the compatibility of multiprocess in python 3.14. since in python 3.14, the default start method of multiprocess has changed from fork to forkserver. So, in the multiprocess tests in pselect.pyx, we can not get the signal of SIGCHLD. It will cause the timeout problem in linux. But in macos, the forkserver has different behavior with the behavior in linux. Maybe the wait_for_process is not useful. I just want to monitor the process sentinel file descriptor instead of SIGCHLD.
In python 3.14, since the cpython refactor the way to count the refer numbers, I replaced direct ob_refcnt access with _Py_REFCNT() macro from cpython.ref

…ork' Python 3.14 changed the default multiprocessing start method from 'fork' to 'forkserver' on Linux, which breaks SIGCHLD signal delivery in tests. This causes pselect tests to hang indefinitely. Changes: - Add Python 3.14 detection in conftest.py - Force multiprocessing to use 'fork' method for Python 3.14+ - Add defensive check for __test__ attribute before deletion Fixes test timeouts on Python 3.14.0rc2. All tests now pass in ~8 seconds (previously timed out at 300s+).

…ndlers Python 3.14 changed recursion checking to use actual C stack pointers instead of counters. The old implementation called Python code from within signal handlers before siglongjmp(), causing _Py_CheckRecursiveCall() to see inconsistent stack pointers and report 'Unrecoverable stack overflow'. Solution: Never call Python code from within signal handlers. Changes: - Added signal_to_raise field to cysigs_t to store deferred signal - Modified signal handlers to only set flag and longjmp (no Python calls) - Added _sig_raise_deferred() to raise exceptions after longjmp - Updated _sig_on_postjmp() to call _sig_raise_deferred() in safe context This maintains async-signal-safety and ensures Python's stack tracking remains consistent. All 65 tests pass on Python 3.14.0rc3. Also updates: - pyproject.toml: requires-python = '>=3.9,<3.15' (was <3.14) - CI/CD: Add Python 3.14 to test matrix

cxzhong · 2025-10-02T14:30:32Z

@tobiasdiez I have tried to fix the signal handle in python 3.14 https://github.com/cxzhong/cysignals/actions/runs/18195804743
And I think there are still problems when building in python 3.14t

- Remove forced fork() from conftest.py to allow Python 3.14's default forkserver multiprocessing start method - Add PSelecter.wait_for_process() method that monitors process.sentinel instead of SIGCHLD, working with any start method (fork/spawn/forkserver) - Update doctests to demonstrate both SIGCHLD-based (fork-only) and sentinel-based (universal) process monitoring approaches - Note: wait_for_process() is POSIX-only; Windows should use multiprocessing.connection.wait() instead This allows cysignals to work properly with Python 3.14's new forkserver default while maintaining backward compatibility.

…refcnt - Replace direct ob_refcnt access with _Py_REFCNT() macro from cpython.ref - Add NULL checks before calling _Py_REFCNT() to prevent null pointer dereference - This fixes compilation errors in Python 3.14t (free-threaded/nogil) build where ob_refcnt is not directly accessible in PyObject structure

…_REFCNT - Remove _Py_REFCNT from cpython.ref import (not available in Cython) - Declare Py_REFCNT as external C function from Python.h - Replace _Py_REFCNT() calls with Py_REFCNT() (2 occurrences) - This properly supports Python 3.14t free-threaded build where ob_refcnt is not directly accessible in PyObject structure

cxzhong · 2025-10-03T08:15:38Z

fix #232 and timeout problem in Linux. and I have finally fixed the problems building in python3.14t. The timeout problem is caused by changing of default start method of python 3.14 multiprocess from fork to forkserver.

tobiasdiez

Thanks a lot!

Mostly looks good to me, based on my superficial understanding of the code. @dimpase @ptrrsn @user202729 could you please have a look at this PR as well?

src/cysignals/signals.pyx

src/conftest.py

cxzhong · 2025-10-04T09:03:36Z

@tobiasdiez can you let the workflows run? Thank you.

Remove unused import of sys module.

Replaced usage of Py_REFCNT with _Py_REFCNT for compatibility with Python 3.14t free-threaded builds.

cxzhong · 2025-10-04T10:04:47Z

Now all tests pass. https://github.com/cxzhong/cysignals/actions/runs/18242832091

user202729 · 2025-10-04T10:30:22Z

instead of adding signal_to_raise, since it's already known by the jmpret value, I suggest just passing that to sig_raise_exception.

The less states there are, the less likely it is to get inconsistent.

Is there any situation where jmpret > 0 yet we don't want to call do_raise_exception?

What does wait_for_process do? It isn't used anywhere.

user202729 · 2025-10-04T10:33:04Z

also, disclose your extent of usage of AI based tools. (most human don't write the pull request description the way you did.)

cxzhong · 2025-10-04T11:03:14Z

also, disclose your extent of usage of AI based tools. (most human don't write the pull request description the way you did.)

I have rewrited the PR description.

Eliminate redundant signal state by using only sigsetjmp/siglongjmp return values to propagate signal numbers. This simplifies the signal handling architecture and prevents potential desynchronization bugs. Changes: - Remove signal_to_raise field from cysigs_t structure - Update signal handlers to not store signal number in global variable - Remove _sig_raise_deferred() function - Modify _sig_on_postjmp() to directly use jump return value - Update Cython declarations accordingly Benefits: - Eliminates redundant state (16 lines removed) - Prevents state desynchronization issues - Signal number propagates naturally via jump mechanism - All 66 tests pass successfully The refactoring maintains full backward compatibility with no functional changes to the API or behavior.

src/cysignals/signals.pxd

This reverts commit b128872.

user202729 · 2025-10-04T18:02:13Z

any idea if this affect anything on WIN32? Given that the siglongjmp is not called on WIN32.

edit: looks like tests pass, so it probably can't be too bad.

cxzhong · 2025-10-04T18:15:05Z

any idea if this affect anything on WIN32? Given that the siglongjmp is not called on WIN32.

edit: looks like tests pass, so it probably can't be too bad.

Maybe we need to test the compatibility in sagemath. To see everything is OK. Thank you very much.

dimpase · 2025-10-04T19:05:33Z

for SageMath on Python 3.14 we need to fix atexit.pyx first, before we move on to other things

… naming conventions

user202729 · 2025-10-05T13:29:58Z

what does wait_for_process do, again?
in which situation may __test__ be undefined?
I said "disclose your AI usage" instead of "rewrite the pull request description". some AI usage are tine, just disclose to which extent.
I've no idea what the change to uv.lock do, ask @tobiasdiez .
otherwise no serious issue.

tobiasdiez · 2025-10-05T13:51:41Z

uv updates are fine.

cxzhong · 2025-10-05T13:54:23Z

what does wait_for_process do, again?

in which situation may __test__ be undefined?

I said "disclose your AI usage" instead of "rewrite the pull request description". some AI usage are tine, just disclose to which extent.

I've no idea what the change to uv.lock do, ask @tobiasdiez .

otherwise no serious issue.

I will delete this. Soon I will push a commit. It seems useless. I just wrote to verify that I do not use the signal SIGCHLD
I am sorry that I do not know that.
I just used GPT5 to analysis the dangerous behavior and get feedback of an ideal, then I just wrote and let the GitHub copilot verify this. and I also use copilot in VS code to summarize the context of commits.
It is generated by uv automatically.
Thank you very much for your feedback

Removed the wait_for_process method that monitored process exit using a sentinel file descriptor.

cxzhong · 2025-10-06T08:28:11Z

I have finished tests on nogil python 3.14t in Linux, it can pass all tests if we run pytest.

Updated Python version in CI workflow from '3.14.0-rc.3' to '3.14'.

Copilot

Pull Request Overview

Align cysignals with Python 3.14 and Cython 3.1, avoiding Python calls in signal handlers, adapting to multiprocessing start-method changes, and updating refcount access for CPython changes.

Defer exception raising from signal handlers; use siglongjmp to return and raise in a safe context.
Replace direct ob_refcnt with _Py_REFCNT and add Cython free-threading compatibility directives; bump Python/Cython minimums.
Update docs/tests to use multiprocessing.get_context and process sentinel; refresh CI runners and Python versions.

Reviewed Changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/cysignals/tests.pyx	Add Cython free-threading compatibility directive.
src/cysignals/signals.pyx	Use _Py_REFCNT and import; add extern for _do_raise_exception.
src/cysignals/signals.pxd	Declare _do_raise_exception (currently under a nogil block).
src/cysignals/pysignals.pyx	Add Cython free-threading compatibility directive.
src/cysignals/pselect.pyx	Doctest updates to use multiprocessing.get_context and sentinels.
src/cysignals/macros.h	Raise exception in safe context via _do_raise_exception in _sig_on_postjmp.
src/cysignals/implementation.c	Avoid Python calls in signal handlers; rename do_raise_exception to _do_raise_exception.
src/cysignals/alarm.pyx	Add Cython free-threading compatibility directive.
src/conftest.py	Safely delete test only if present.
pyproject.toml	Bump Cython/Python requirements; update classifiers.
README.rst	Update requirements to Python ≥3.11 and Cython ≥3.1.
.github/workflows/dist.yml	Update macOS runner to 14.
.github/workflows/ci.yml	Add Python 3.14 to matrix; update macOS runner.

Comments suppressed due to low confidence (3)

src/cysignals/implementation.c:1

On Windows, the signal handler no longer raises an exception or returns via a trampoline; the previous do_raise_exception(sig) call was removed and there's no alternative in the _WIN32 path. Add an #else branch that calls _do_raise_exception(sig) on Windows to avoid losing interrupts.

/*

src/cysignals/implementation.c:1

Same issue for non-SIGINT signals: on Windows, the handler now does neither a longjmp nor raises an exception. Add an _WIN32 branch that calls _do_raise_exception(sig) to preserve behavior on Windows.

/*

src/cysignals/signals.pxd:1

_do_raise_exception raises Python exceptions and must run with the GIL, but it's declared inside a cdef nogil block, permitting calls without the GIL. Move this declaration out of the nogil block (declare it under a normal cdef:) or ensure _do_raise_exception acquires/releases the GIL internally in all code paths.

# cython: preliminary_late_includes_cy28=True

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-08T08:01:32Z

src/cysignals/implementation.c

 static void cysigs_signal_handler(int sig);

-static void do_raise_exception(int sig);
+static void _do_raise_exception(int sig);


_do_raise_exception is declared static here, but it is invoked from macros.h's inline _sig_on_postjmp, which is compiled into other translation units. With static linkage, other TUs cannot resolve this symbol, causing link errors. Remove static (make it have external linkage) and provide a prototype in a shared header (e.g., macros.h) so all TUs see the declaration.

Suggested change

static void _do_raise_exception(int sig);

void _do_raise_exception(int sig);

Copilot · 2025-10-08T08:01:33Z

src/cysignals/macros.h

+        /* A signal occurred and we jumped back via longjmp.
+         * jmpret contains the signal number that was passed to siglongjmp.
+         * Now we're back in a safe context (not in signal handler),
+         * so it's safe to call Python code to raise the exception. */
+        _do_raise_exception(jmpret);


This inline function calls _do_raise_exception but there's no C declaration visible in this header. Without an extern prototype, this will be an implicit declaration (or an error under modern compilers). Add an extern prototype for _do_raise_exception(int) in a header included by all consumers (e.g., declare it at the top of macros.h) and ensure the function has external linkage.

Copilot · 2025-10-08T08:01:33Z

src/cysignals/macros.h

 * Return 0 if there was an exception, 1 otherwise.
+ *
+ * This function propagates the signal number naturally via jmpret
+ * (the return value from sigsetjmp/cylongjmp), which is always nonzero


Correct 'cylongjmp' to 'siglongjmp'.

Suggested change

* (the return value from sigsetjmp/cylongjmp), which is always nonzero

* (the return value from sigsetjmp/siglongjmp), which is always nonzero

.github/workflows/ci.yml

Co-authored-by: Copilot <[email protected]>

cxzhong and others added 3 commits October 1, 2025 16:16

CI: Use specific Python 3.14.0-rc.3 version for testing

1389e13

cxzhong force-pushed the main branch from cdeb03f to 1389e13 Compare October 3, 2025 05:21

cxzhong added 2 commits October 3, 2025 13:26

cxzhong force-pushed the main branch 2 times, most recently from 262520b to 546217f Compare October 3, 2025 05:51

tobiasdiez reviewed Oct 4, 2025

View reviewed changes

src/cysignals/signals.pyx Outdated Show resolved Hide resolved

src/conftest.py Outdated Show resolved Hide resolved

cxzhong added 2 commits October 4, 2025 17:37

Remove unused sys import from conftest.py

68b7713

Remove unused import of sys module.

Update reference count check for cysigs.exc_value

c298389

Replaced usage of Py_REFCNT with _Py_REFCNT for compatibility with Python 3.14t free-threaded builds.

user202729 reviewed Oct 4, 2025

View reviewed changes

src/cysignals/signals.pxd Outdated Show resolved Hide resolved

cxzhong and others added 2 commits October 4, 2025 23:37

Remove do_raise_exception declaration from signals.pxd

b128872

Revert "Remove do_raise_exception declaration from signals.pxd"

1c1bcfd

This reverts commit b128872.

cxzhong requested a review from tobiasdiez October 4, 2025 17:52

cxzhong changed the title ~~fix compatible in python 3.14~~ fix compatible issues in python 3.14 Oct 4, 2025

cxzhong added 2 commits October 5, 2025 15:33

Update macOS version in CI workflow

e119a0c

Rename do_raise_exception to _do_raise_exception for consistency with…

33533c5

… naming conventions

cxzhong requested a review from user202729 October 5, 2025 11:18

cxzhong added 4 commits October 5, 2025 21:57

Remove wait_for_process method from pselect.pyx

385d66c

Removed the wait_for_process method that monitored process exit using a sentinel file descriptor.

Update macOS version in dist workflow

cf49a75

Add freethreading compatibility declaration to Cython files

7c92255

Merge branch 'main' of github.com:cxzhong/cysignals

8f32f8b

cxzhong added 3 commits October 8, 2025 09:51

Update Python version in CI workflow

ba28e30

Updated Python version in CI workflow from '3.14.0-rc.3' to '3.14'.

Update Python version requirements in pyproject.toml

d6bc361

change pyproject.toml and uv.lock and modify redame

7af95ed

cxzhong marked this pull request as ready for review October 8, 2025 07:56

Copilot AI review requested due to automatic review settings October 8, 2025 07:56

Remove unused

b5d71e9

Copilot AI reviewed Oct 8, 2025

View reviewed changes

Update .github/workflows/ci.yml

105ff80

Co-authored-by: Copilot <[email protected]>

	static void _do_raise_exception(int sig);
	void _do_raise_exception(int sig);

	* (the return value from sigsetjmp/cylongjmp), which is always nonzero
	* (the return value from sigsetjmp/siglongjmp), which is always nonzero

Uh oh!

fix compatible issues in python 3.14 #236

Are you sure you want to change the base?

fix compatible issues in python 3.14 #236

Conversation

cxzhong commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cxzhong commented Oct 2, 2025

Uh oh!

cxzhong commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tobiasdiez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cxzhong commented Oct 4, 2025

Uh oh!

cxzhong commented Oct 4, 2025

Uh oh!

user202729 commented Oct 4, 2025

Uh oh!

user202729 commented Oct 4, 2025

Uh oh!

cxzhong commented Oct 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

user202729 commented Oct 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cxzhong commented Oct 4, 2025

Uh oh!

dimpase commented Oct 4, 2025

Uh oh!

user202729 commented Oct 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tobiasdiez commented Oct 5, 2025

Uh oh!

cxzhong commented Oct 5, 2025

Uh oh!

cxzhong commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cxzhong commented Oct 2, 2025 •

edited

Loading

cxzhong commented Oct 3, 2025 •

edited

Loading

cxzhong commented Oct 4, 2025 •

edited

Loading

user202729 commented Oct 4, 2025 •

edited

Loading

user202729 commented Oct 5, 2025 •

edited

Loading

cxzhong commented Oct 6, 2025 •

edited

Loading