Prevent greenlet from building on free-threaded Python

[jamadden]

We don't set the flag that indicates we're compatible with free-threading, so loading us would automatically enable the GIL according to the Python documentation. It is thus misleading that we can even build on a free-threaded Python and get the "t" suffix, e.g., "313t".

The make-manylinux script has some support for skipping free-threaded Python's, but it's neither complete nor sufficient.

The trouble is that CPython doesn't document a way to detect at compile time that it is built on free-threading. At least, not that I could see in the "Porting to 3.13" and "C API Changes" sections of the release notes.

[LeroyK111]

0.06s - pydevd: GEVENT_SUPPORT is set but gevent is not available in the environment. Please unset GEVENT_SUPPORT from the environment variables or install gevent. Traceback (most recent call last): File "c:\Users\Administrator\.vscode\extensions\ms-python.debugpy-2024.13.2024101501-win32-x64\bundled\libs\debugpy\_vendored\pydevd\pydevd.py", line 166, in <module> from _pydevd_bundle import pydevd_gevent_integration File "c:\Users\Administrator\.vscode\extensions\ms-python.debugpy-2024.13.2024101501-win32-x64\bundled\libs\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_gevent_integration.py", line 2, in <module> import greenlet File "d:\python\Lib\site-packages\greenlet\__init__.py", line 29, in <module> from ._greenlet import _C_API # pylint:disable=no-name-in-module ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'greenlet._greenlet'

So, I configured the launch.json file in VSCode and set the "gevent" field to false. After that, I was able to use Python's free-threaded version, which disables the GIL (Global Interpreter Lock) without any issues.

[jamadden]

If you load the extension, it's going to initialize the GIL. If you force Python to not initialize the GIL, the extension will be broken. Do not do this.

[ngoldbaum]

The trouble is that CPython doesn't document a way to detect at compile time that it is built on free-threading. At least, not that I could see in the "Porting to 3.13" and "C API Changes" sections of the release notes.

You can check if Py_GIL_DISABLED is set.

Thinking further afield - is there an issue for supporting the free-threaded build? I'm actively working on adding support for free-threaded Python in community packages and can offer help and advice. There's also https://py-free-threading.github.io which has some advice for porting extensions.

[jamadden]

You can check if Py_GIL_DISABLED is set.

So I just need:

// greenlet.h 
#ifdef Py_GIL_DISABLED
#error Nope right out of that.
#endif       

is there an issue for supporting the free-threaded build?

Unfortunately yes. greenlet itself implicitly depends on the GIL to keep its internal structures consistent. We could add a bunch of locks internally to take care of that, but that leaves bigger issues. I'm not saying they can't be dealt with, but they are certainly factors that make trying to support GILless CPython risky.

Bigger issue #1 is that greenlet copies and mutates internal CPython state and then restores it, and that data must be completely consistent. That state should all belong to one thread, and shouldn't be manipulated by other threads, but that's a lot of wishful thinking that I don't know is actually true, especially when it comes to internal CPython data structures that can be manipulated as side-effects (e.g., there has to be a list of threads and thread creating and destruction has to manipulate that list; if something we copy/restore during greenlet switching refers to that list, concurrent manipulation of it could be deadly). Extensive testing would be required (and its hard to test for race conditions.)

Bigger issue #2 is effectively the same issue that the Python community as a whole has to deal with --- race conditions that have been hidden until now --- only writ large. Because greenlets are fully cooperative, users are used to ignoring race conditions altogether. Already when you try to mix threads with greenlets things get dicy --- gevent has a lot of code to try to make this safe and workable, but it's not perfect --- and actually adding true concurrency on top of that is only going to magnify the problem. (Oh, and also, gevent depends on the GIL for some of its invariants as well, so gevent would have to be reworked too.) Again, not an insurmountable problem (and technically not one that greenlet can really do anything about), but a factor to consider before deciding if it's worth spending the effort.

[ngoldbaum]

So I just need:

You need to #include "python.h" so if greenlet.h does that then yes.

Thanks for the context!

I'm not sure whether pip will continue to install the cp313t wheels on pypi if there is a newer version without cp313t wheels.

[grantlouisherman]

Hey all has there been any progress on this issue?

[LeroyK111]

Hey all has there been any progress on this issue?

Unfortunately, there has been no real progress on this. The recommended workaround is still to disable GEVENT_SUPPORT.
The situation with Python has become even more problematic, since newer versions (like 3.14) break gevent/greenlet compatibility at the C API level.

[jamadden]

since newer versions (like 3.14) break gevent/greenlet compatibility at the C API level.

Not true. gevent and greenlet both support Python 3.14.

[zzzeek]

you can run greenlet with freethreading python without the GIL; just set PYTHON_GIL=0:

$ ~/.venv313t/bin/python
Python 3.13.7 experimental free-threading build (main, Sep 26 2025, 12:29:58) [GCC 14.3.1 20250523 (Red Hat 14.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import greenlet
<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'greenlet._greenlet', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

Greenlet does work, a least for a little while, in this mode:

$ PYTHON_GIL=0 ~/.venv313t/bin/python
Python 3.13.7 experimental free-threading build (main, Sep 26 2025, 12:29:58) [GCC 14.3.1 20250523 (Red Hat 14.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import greenlet
>>> def run():
...     print("Running in the greenlet function.")
>>>  glet = greenlet.greenlet(run)
>>> glet.switch()
Running in the greenlet function.

however, running big things like SQLAlchemy's test suite with full freethreading + greenlet wrappers for tests, the suite crashes about halfway through.

Unfortunately yes. greenlet itself implicitly depends on the GIL to keep its internal structures consistent. We could add a bunch of locks internally to take care of that, but that leaves bigger issues. I'm not saying they can't be dealt with, but they are certainly factors that make trying to support GILless CPython risky.

Here's the thing. free threading is about to be officially supported and at some point likely sooner than later is going to be the default. So I think this issue is going to become existential for greenlet (and, selfishly, SQLAlchemy's entire asyncio API) if it doesn't try to tackle this problem in some way.

Bigger issue #1 is that greenlet copies and mutates internal CPython state and then restores it, and that data must be completely consistent. That state should all belong to one thread, and shouldn't be manipulated by other threads, but that's a lot of wishful thinking that I don't know is actually true, especially when it comes to internal CPython data structures that can be manipulated as side-effects

how does GIL-less impact that? GIL python would have the same question to be answered no?

Bigger issue #2 is effectively the same issue that the Python community as a whole has to deal with --- race conditions that have been hidden until now --- only writ large. Because greenlets are fully cooperative, users are used to ignoring race conditions altogether. Already when you try to mix threads with greenlets things get dicy --- gevent has a lot of code to try to make this safe and workable, but it's not perfect --- and actually adding true concurrency on top of that is only going to magnify the problem.

Unfortunately that's what it means to support free threading, that test suites are continued to be enhanced to find more of these races and fixes are added. It's work but it is a path forward. In SQLAlchemy I have already added my own mini-gil which I applied to a few spots to overcome some of these issues as they were encountered, but lucky me these were pure python race conditions, not C code.

nevertheless I think greenlet is going to need to tackle this problem in the next year or so if not sooner because I think free-threading might end up being popular.

[jamadden]

I too agree that it's time to address this by adding free-threading support. I very much look forward to the day when that's the default. It can almost certainly be done with a little bit of work. I hope to find the time soon.

Bigger issue #1 is that greenlet copies and mutates internal CPython state and then restores it

how does GIL-less impact that? GIL python would have the same question to be answered no?

No, because the GIL ensures we see a consistent internal state. Without being able to use the same lock CPython itself uses when manipulating internal data structures (which lock, again, is currently the GIL) we don't necessarily have that guarantee. I haven't looked into it any further yet.