Add Python 3.13 and 3.14 support#28
Open
godlygeek wants to merge 4 commits into
Open
Conversation
godlygeek
force-pushed
the
add-python-3.13-3.14-support
branch
3 times, most recently
from
8f31cb7 to
b8766d8
Compare
Contributor
Author
|
And, unfortunately, it turns out that there were a bunch of other changes needed on top of that, to correctly cope with Python 3.14 changing the default multiprocessing start method on Linux from "fork" to "forkserver". I think I've got them all ironed out now, though. |
Since Python 3.14 'forkserver' is now the default multiprocessing start method instead of 'fork', and with this spawn method we need to pass the `multiprocessing.Event` that we're sharing between the pytest process and the monitoring process along explicitly, rather than counting on a `fork()` to share the same object between the two processes.
Python 3.14 defaults to the "forkserver" multiprocessing start method, instead of the previous "fork" default. This new default causes all sorts of problems for us. The hardest to fix one, which made me throw in the towel on trying to get things working with "forkserver", is that pytest's `pytester` fixture defaults to running the new pytest session in the current process, rather than from a subprocess, and we're using `capfd` to capture the stderr. With the "fork" method and the "spawn" method, the new process inherits whatever the parent process's stderr (fd 2) is pointed to at the time when the new process is started, but with "forkserver" the server gets created once and then reused, and all future children have their stderr pointed wherever the first child process's stderr was pointed to. There's other issues too. We leak semaphores because the monitoring process winds up creating an unnecessary `Event` and `Queue` that it ignores in favor of the ones passed to its entry point. That could be fixed by creating those lazily instead of at import time, but that leads to pickling failures because the `multiprocessing.connection` module gets created lazily, and `pytester` unloads modules that were loaded by the tests, which leads to two different copies of that module getting loaded, with the old module still referenced by some things. We could work around all of this in our test suite, but using our plugin along with the `pytester` fixture is a reasonable use case that end users might want, if they're trying to use `pytest-pystack` to test their own pytest plugin. The most reasonable solution is for us to detect when "forkserver" would be used, and to use "spawn" instead in those cases.
godlygeek
force-pushed
the
add-python-3.13-3.14-support
branch
from
b8766d8 to
ef2ee8e
Compare
Contributor
Author
|
@pablogsal I'd appreciate your eyes here. This turned out to be shockingly tricky, and I'd appreciate a double check to make sure there's nothing else I'm missing. |
Rather than trying to support both "fork" and "spawn", force "spawn" to be used no matter what. Split the monitor process's entry point out into a separate module to avoid having a global multiprocessing `Queue` object get created when it is reimported in the spawned process, as otherwise Python 3.8 and 3.9 report that some semaphores are leaked by the child process.
lkollar
approved these changes
Jun 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #15
This resurrects #16, which slipped through the cracks and got closed without ever being merged. Sorry about that, @zeel2104!
I've rebased it on main and dropped a few diff hunks that we don't want. Most importantly, I want to keep the
python-version: ${{ matrix.python_version }}-dev(we don't need the-devsuffix right now, but we'll need it again soon when we add 3.15 support). Other than that, I reverted some changes to things that were already addressed by other PRs.