diff --git a/peps/pep-0780.rst b/peps/pep-0780.rst index b488e3872cc..bd0b8441557 100644 --- a/peps/pep-0780.rst +++ b/peps/pep-0780.rst @@ -8,7 +8,7 @@ Status: Draft Type: Standards Track Topic: Packaging Created: 21-Mar-2025 -Python-Version: 3.14 +Python-Version: 3.15 Post-History: `05-Aug-2024 `__, `26-Mar-2025 `__, @@ -22,15 +22,16 @@ Post-History: Abstract ======== -This PEP defines using ABI features as environment markers for project -dependencies, through a new ``sys_abi_features`` environment marker. :pep:`508` -(later moved to :ref:`packaging:dependency-specifiers`) introduced environment -markers to specify dependencies based on rules that describe when the -dependency should be used. This PEP extends the environment markers to allow -specifying dependencies based on specific ABI features of the Python -interpreter. For this, it defines a set of `ABI Features`_ and specifies how -they are made available for `environment markers `_ -as a new marker variable, ``sys_abi_features``. +This PEP defines the use of information about the interpreter ABI as +environment markers for project dependencies through a new ``sys_abi_info`` +environment marker variable. :pep:`508` (later moved to +:ref:`packaging:dependency-specifiers`) introduced environment markers to +specify dependencies based on rules that describe when the dependency should +be used. This PEP extends environment markers to allow specifying dependencies +based on specific information about the ABI of the Python interpreter. For +this, it defines a set of `ABI Features`_ and specifies how they are made +available for `environment markers `_ via a new marker +variable, ``sys_abi_info``. Motivation ========== @@ -57,23 +58,19 @@ marker for free-threading`_ Discourse thread: wheels. Picking up the wrong Cython version is causing a lot of obscure build failures and runtime crashes. It would be beneficial if the metadata could express that (c.f. `Require Cython Pre-release for Free-Threaded Python`_). -- CFFI has no support for free-threading yet, and `Armin Rigo, one of the - maintainers, has stated - `__ - that it may be a good idea to fork ``cffi``, implement support for - free-threading, and only come back to the CFFI project with a single large PR - that adds support after the functionality "is reasonably well-tested (either - as tests or, better in this case, tested by being in use in various other - projects)". There are a lot of projects that depend on ``cffi``. They are - likely fine to start depending on a fork for free-threading only, however +- CFFI has only recently gained support for free-threading, and for a + significant period of time, it looked likely that support would first be + added in a fork. There are a lot of projects that depend on ``cffi``, which + are likely fine to start depending on a fork for free-threading only, however depending on a fork for >=3.13 or for all Python versions seems like a much - larger ask, and more disruptive for distribution packagers. + larger ask, and more disruptive for distribution packagers. While the + situation has been resolved with CFFI 2.0.0, other projects may face similar + issues. -While these concrete examples may be addressed later this year by Cython and -CFFI making compatible releases, the same issue is going to repeat further up -the stack. The free-threading rollout is expected to take several years, and an -environment marker for free-threading will make that rollout significantly -easier. +While concrete examples may be addressed with compatible releases, the same +issue is going to repeat further up the stack. The free-threading rollout is +expected to take several years, and an environment marker for free-threading +will make that rollout significantly easier. Another important ABI feature that is not yet covered by environment markers is the bitness of the interpreter. In most cases, the ``sys_platform`` or @@ -135,7 +132,7 @@ Relation to Other PEPs This PEP extends environment markers with set semantics for ABI features. :pep:`751#additions-to-marker-expression-syntax` includes a similar extension for lock file specific environment markers; although the two have been -developed indepedently, they are compatible where they overlap in terms of the +developed independently, they are compatible where they overlap in terms of the new set semantics. Specification @@ -149,49 +146,55 @@ and "**OPTIONAL**"" in this document are to be interpreted as described in ABI Features ------------ -ABI features are intrinsic properties of the Python interpreter, expressed as -simple, understandable strings. However, not all features are equally -applicable to all Python interpreters or Python versions. For example, the -distinction between free-threaded and GIL-enabled interpreters is only relevant -for CPython 3.13 onwards, but the bitness of the interpreter is relevant for -all interpreters. +ABI features are intrinsic properties of the Python interpreter. +However, not all features are equally applicable to all Python interpreters or +Python versions. For example, the distinction between free-threaded and +GIL-enabled interpreters is only relevant for CPython 3.13 onwards, but the +bitness of the interpreter, i.e. the number of bits in a pointer, is relevant +for all interpreters. All interpreters MUST handle the following ABI features as stated. ABI features that are restricted to particular interpreters MUST NOT be provided by other -interpreters. The features are subdivided into groups and for each group there -MUST be exactly one feature present, except when the group is marked as -optional, in which case there MUST be at most one feature present. +interpreters. -``free-threading`` or ``gil-enabled`` (only CPython) - If the Python interpreter is free-threaded, the ``free-threading`` feature - MUST be present and the ``gil-enabled`` feature MUST NOT be present. - Otherwise, the ``gil-enabled`` feature MUST be present and the - ``free-threading`` feature MUST NOT be present. +In Python 3.15 and later, the ABI features are determined by the new +:external+py3.15:py:data:`sys.abi_info` attribute. In earlier Python versions, +the ABI features are the same as for Python 3.15 and given as follows: -``debug`` (only CPython, optional) - This ABI feature is reserved for the ``--with-pydebug`` build of CPython. - If the interpreter is a CPython interpreter with ``Py_DEBUG`` capabilities, - the ``debug`` feature MUST be present. On POSIX systems, this corresponds - to the Python expression ``"d" in sys.abiflags``. +``pointer_bits`` + The bitness of the interpreter, that is, the number of bits in a pointer, + typically ``32`` or ``64`` [#bitness]_. -``32-bit`` or ``64-bit`` (optional) - The bitness of the interpreter, that is, whether it is a 32-bit or 64-bit - build [#bitness]_. If the bitness is unknown or neither 32-bit nor 64-bit, - this feature MUST NOT be present. +``free_threaded`` (only CPython) + If the Python interpreter is free-threaded, ``free_threaded`` MUST be + ``True``, otherwise it MUST be ``False``. -.. _pep-780-sys_abi_features: +``debug`` (only CPython) + This ABI feature corresponds to the + :external+py3.15:py:attr:`sys.abi_info.debug` field. -The ``sys_abi_features`` Environment Marker -------------------------------------------- +``byteorder`` + Indicates the byte order of the interpreter. The value MUST be either + ``"little"`` or ``"big"``. + +.. _pep-780-sys_abi_info: + +The ``sys_abi_info`` Environment Marker +--------------------------------------- To make ABI features available in dependency specifications, a new environment -marker variable, ``sys_abi_features``, is added to the format of dependency -specifiers. +marker variable, ``sys_abi_info``, is added to the format of dependency +specifiers. It is built to reflect the value of ``sys.abi_info``, which is a +new attribute of the ``sys`` module introduced in Python 3.15 as a simple +namespace. To represent its fields in the environment marker variable, they are +expressed as a frozen set of strings of the form ``f"{name}::{value}"``, where +``name`` is the name of the field and ``value`` is the string representation of +its value. To do this, we need to extend the grammar laid out in :pep:`508` and maintained in the :ref:`packaging:dependency-specifiers` and document the possible values. -The grammar is extended to include the ``sys_abi_features`` marker variable by +The grammar is extended to include the ``sys_abi_info`` marker variable by augmenting the definition of ``env_var`` as follows:: env_var = ('python_version' | 'python_full_version' | @@ -199,7 +202,7 @@ augmenting the definition of ``env_var`` as follows:: 'platform_system' | 'platform_version' | 'platform_machine' | 'platform_python_implementation' | 'implementation_name' | 'implementation_version' | - 'sys_abi_features' | + 'sys_abi_info' | 'extra' # ONLY when defined by a containing layer ) @@ -212,10 +215,10 @@ Like the grammar, also the overview table of environment markers in * - Marker - Python equivalent - Sample values - * - ``sys_abi_features`` - - no direct equivalent available - - ``{'free-threading', '64-bit'}``, - ``{'gil-enabled', 'debug', '32-bit'}`` + * - ``sys_abi_info`` + - ``sys.abi_info`` + - ``{'free_threaded::True', 'pointer_bits::64'}``, + ``{'free_threaded::False', 'debug::True', 'pointer_bits::32'}`` With these additions, ABI features can be used in dependency specifications via the ``in`` operator to test for the presence of a feature, or the ``not in`` @@ -229,22 +232,22 @@ Require Cython Pre-release for Free-Threaded Python To require a pre-release of Cython only for a free-threaded Python interpreter, the following dependency specification can be used:: - cython >3.1.0a1; "free-threading" in sys_abi_features - cython ==3.0.*; "free-threading" not in sys_abi_features + cython >3.1.0a1; "free_threaded::True" in sys_abi_info + cython ==3.0.*; "free_threaded::False" in sys_abi_info Require SciPy Unless on 32-bit ``win32`` ---------------------------------------- To require SciPy unless on a 32-bit interpreter on Windows, the following dependency specification can be used:: - scipy; platform_system != "Windows" or "32-bit" not in sys_abi_features + scipy; platform_system != "Windows" or "pointer_bits::32" not in sys_abi_info Require NumPy for a Free-Threaded Interpreter With Debugging Capabilities -------------------------------------------------------------------------- To require NumPy only for a free-threaded interpreter with debugging capabilities, the following dependency can be used:: - numpy; "free-threading" in sys_abi_features and "debug" in sys_abi_features + numpy; "free-threaded::True" in sys_abi_info and "debug::True" in sys_abi_info Backwards Compatibility =======================