Merge branch 'main' into posix-spawn-test

Author: jxes993409

Doc/c-api/module.rst

@@ -388,6 +388,28 @@ The available slot types are:
 
    .. versionadded:: 3.13
 
+.. c:macro:: Py_mod_abi
+
+   A pointer to a :c:struct:`PyABIInfo` structure that describes the ABI that
+   the extension is using.
+
+   When the module is loaded, the :c:struct:`!PyABIInfo` in this slot is checked
+   using :c:func:`PyABIInfo_Check`.
+
+   A suitable :c:struct:`!PyABIInfo` variable can be defined using the
+   :c:macro:`PyABIInfo_VAR` macro, as in:
+
+   .. code-block:: c
+
+      PyABIInfo_VAR(abi_info);
+
+      static PyModuleDef_Slot mymodule_slots[] = {
+         {Py_mod_abi, &abi_info},
+         ...
+      };
+
+   .. versionadded:: 3.15
+
 
 .. _moduledef-dynamic:
 

Doc/c-api/stable.rst

@@ -2,9 +2,9 @@
 
 .. _stable:
 
-***************
-C API Stability
-***************
+***********************
+C API and ABI Stability
+***********************
 
 Unless documented otherwise, Python's C API is covered by the Backwards
 Compatibility Policy, :pep:`387`.
@@ -199,6 +199,162 @@ This is the case with Windows and macOS releases from ``python.org`` and many
 third-party distributors.
 
 
+ABI Checking
+============
+
+.. versionadded:: next
+
+Python includes a rudimentary check for ABI compatibility.
+
+This check is not comprehensive.
+It only guards against common cases of incompatible modules being
+installed for the wrong interpreter.
+It also does not take :ref:`platform incompatibilities <stable-abi-platform>`
+into account.
+It can only be done after an extension is successfully loaded.
+
+Despite these limitations, it is recommended that extension modules use this
+mechanism, so that detectable incompatibilities raise exceptions rather than
+crash.
+
+Most modules can use this check via the :c:data:`Py_mod_abi`
+slot and the :c:macro:`PyABIInfo_VAR` macro, for example like this:
+
+.. code-block:: c
+
+   PyABIInfo_VAR(abi_info);
+
+   static PyModuleDef_Slot mymodule_slots[] = {
+      {Py_mod_abi, &abi_info},
+      ...
+   };
+
+
+The full API is described below for advanced use cases.
+
+.. c:function:: int PyABIInfo_Check(PyABIInfo *info, const char *module_name)
+
+   Verify that the given *info* is compatible with the currently running
+   interpreter.
+
+   Return 0 on success. On failure, raise an exception and return -1.
+
+   If the ABI is incompatible, the raised exception will be :py:exc:`ImportError`.
+
+   The *module_name* argument can be ``NULL``, or point to a NUL-terminated
+   UTF-8-encoded string used for error messages.
+
+   Note that if *info* describes the ABI that the current code uses (as defined
+   by :c:macro:`PyABIInfo_VAR`, for example), using any other Python C API
+   may lead to crashes.
+   In particular, it is not safe to examine the raised exception.
+
+   .. versionadded:: next
+
+.. c:macro:: PyABIInfo_VAR(NAME)
+
+   Define a static :c:struct:`PyABIInfo` variable with the given *NAME* that
+   describes the ABI that the current code will use.
+   This macro expands to:
+
+   .. code-block:: c
+
+      static PyABIInfo NAME = {
+          1, 0,
+          PyABIInfo_DEFAULT_FLAGS,
+          PY_VERSION_HEX,
+          PyABIInfo_DEFAULT_ABI_VERSION
+      }
+
+   .. versionadded:: next
+
+.. c:type:: PyABIInfo
+
+   .. c:member:: uint8_t abiinfo_major_version
+
+      The major version of :c:struct:`PyABIInfo`. Can be set to:
+
+      * ``0`` to skip all checking, or
+      * ``1`` to specify this version of :c:struct:`!PyABIInfo`.
+
+   .. c:member:: uint8_t abiinfo_minor_version
+
+      The major version of :c:struct:`PyABIInfo`.
+      Must be set to ``0``; larger values are reserved for backwards-compatible
+      future versions of :c:struct:`!PyABIInfo`.
+
+   .. c:member:: uint16_t flags
+
+      .. c:namespace:: NULL
+
+      This field is usually set to the following macro:
+
+      .. c:macro:: PyABIInfo_DEFAULT_FLAGS
+
+         Default flags, based on current values of macros such as
+         :c:macro:`Py_LIMITED_API` and :c:macro:`Py_GIL_DISABLED`.
+
+      Alternately, the field can be set to to the following flags, combined
+      by bitwise OR.
+      Unused bits must be set to zero.
+
+      ABI variant -- one of:
+
+         .. c:macro:: PyABIInfo_STABLE
+
+            Specifies that the stable ABI is used.
+
+         .. c:macro:: PyABIInfo_INTERNAL
+
+            Specifies ABI specific to a particular build of CPython.
+            Internal use only.
+
+      Free-threading compatibility -- one of:
+
+         .. c:macro:: PyABIInfo_FREETHREADED
+
+            Specifies ABI compatible with free-threading builds of CPython.
+            (That is, ones compiled with :option:`--disable-gil`; with ``t``
+            in :py:data:`sys.abiflags`)
+
+         .. c:macro:: PyABIInfo_GIL
+
+            Specifies ABI compatible with non-free-threading builds of CPython
+            (ones compiled *without* :option:`--disable-gil`).
+
+   .. c:member:: uint32_t build_version
+
+      The version of the Python headers used to build the code, in the format
+      used by :c:macro:`PY_VERSION_HEX`.
+
+      This can be set to ``0`` to skip any checks related to this field.
+      This option is meant mainly for projects that do not use the CPython
+      headers directly, and do not emulate a specific version of them.
+
+   .. c:member:: uint32_t abi_version
+
+      The ABI version.
+
+      For the Stable ABI, this field should be the value of
+      :c:macro:`Py_LIMITED_API`
+      (except if :c:macro:`Py_LIMITED_API` is ``3``; use
+      :c:expr:`Py_PACK_VERSION(3, 2)` in that case).
+
+      Otherwise, it should be set to :c:macro:`PY_VERSION_HEX`.
+
+      It can also be set to ``0`` to skip any checks related to this field.
+
+      .. c:namespace:: NULL
+
+      .. c:macro:: PyABIInfo_DEFAULT_ABI_VERSION
+
+         The value that should be used for this field, based on current
+         values of macros such as :c:macro:`Py_LIMITED_API`,
+         :c:macro:`PY_VERSION_HEX` and :c:macro:`Py_GIL_DISABLED`.
+
+   .. versionadded:: next
+
+
 .. _limited-api-list:
 
 Contents of Limited API

Doc/conf.py

@@ -245,7 +245,6 @@
     ('py:attr', '__annotations__'),
     ('py:meth', '__missing__'),
     ('py:attr', '__wrapped__'),
-    ('py:meth', 'index'),  # list.index, tuple.index, etc.
 ]
 
 # gh-106948: Copy standard C types declared in the "c:type" domain and C

Doc/data/stable_abi.dat

@@ -591,9 +591,9 @@ exhaustive test suites that exercise every line of code in a module.
 An appropriate testing discipline can help build large complex applications in
 Python as well as having interface specifications would.  In fact, it can be
 better because an interface specification cannot test certain properties of a
-program.  For example, the :meth:`!list.append` method is expected to add new elements
+program.  For example, the :meth:`list.append` method is expected to add new elements
 to the end of some internal list; an interface specification cannot test that
-your :meth:`!list.append` implementation will actually do this correctly, but it's
+your :meth:`list.append` implementation will actually do this correctly, but it's
 trivial to check this property in a test suite.
 
 Writing test suites is very helpful, and you might want to design your code to

Doc/faq/design.rst

@@ -454,7 +454,7 @@ There are two factors that produce this result:
    (the list), and both ``x`` and ``y`` refer to it.
 2) Lists are :term:`mutable`, which means that you can change their content.
 
-After the call to :meth:`!append`, the content of the mutable object has
+After the call to :meth:`~sequence.append`, the content of the mutable object has
 changed from ``[]`` to ``[10]``.  Since both the variables refer to the same
 object, using either name accesses the modified value ``[10]``.
 
@@ -1397,9 +1397,9 @@ To see why this happens, you need to know that (a) if an object implements an
 :meth:`~object.__iadd__` magic method, it gets called when the ``+=`` augmented
 assignment
 is executed, and its return value is what gets used in the assignment statement;
-and (b) for lists, :meth:`!__iadd__` is equivalent to calling :meth:`!extend` on the list
-and returning the list.  That's why we say that for lists, ``+=`` is a
-"shorthand" for :meth:`!list.extend`::
+and (b) for lists, :meth:`!__iadd__` is equivalent to calling
+:meth:`~sequence.extend` on the list and returning the list.
+That's why we say that for lists, ``+=`` is a "shorthand" for :meth:`list.extend`::
 
     >>> a_list = []
     >>> a_list += [1]

Doc/faq/programming.rst

@@ -1251,8 +1251,9 @@ Glossary
       The :class:`collections.abc.Sequence` abstract base class
       defines a much richer interface that goes beyond just
       :meth:`~object.__getitem__` and :meth:`~object.__len__`, adding
-      :meth:`!count`, :meth:`!index`, :meth:`~object.__contains__`, and
-      :meth:`~object.__reversed__`. Types that implement this expanded
+      :meth:`~sequence.count`, :meth:`~sequence.index`,
+      :meth:`~object.__contains__`, and :meth:`~object.__reversed__`.
+      Types that implement this expanded
       interface can be registered explicitly using
       :func:`~abc.ABCMeta.register`. For more documentation on sequence
       methods generally, see

Doc/glossary.rst

@@ -46,6 +46,10 @@ and :func:`call_annotate_function`, as well as the
 :func:`call_evaluate_function` function for working with
 :term:`evaluate functions <evaluate function>`.
 
+.. caution::
+
+   Most functionality in this module can execute arbitrary code; see
+   :ref:`the security section <annotationlib-security>` for more information.
 
 .. seealso::
 
@@ -604,3 +608,23 @@ Below are a few examples of the behavior with unsupported expressions:
    >>> def ifexp(x: 1 if y else 0): ...
    >>> get_annotations(ifexp, format=Format.STRING)
    {'x': '1'}
+
+.. _annotationlib-security:
+
+Security implications of introspecting annotations
+--------------------------------------------------
+
+Much of the functionality in this module involves executing code related to annotations,
+which can then do arbitrary things. For example,
+:func:`get_annotations` may call an arbitrary :term:`annotate function`, and
+:meth:`ForwardRef.evaluate` may call :func:`eval` on an arbitrary string. Code contained
+in an annotation might make arbitrary system calls, enter an infinite loop, or perform any
+other operation. This is also true for any access of the :attr:`~object.__annotations__` attribute,
+and for various functions in the :mod:`typing` module that work with annotations, such as
+:func:`typing.get_type_hints`.
+
+Any security issue arising from this also applies immediately after importing
+code that may contain untrusted annotations: importing code can always cause arbitrary operations
+to be performed. However, it is unsafe to accept strings or other input from an untrusted source and
+pass them to any of the APIs for introspecting annotations, for example by editing an
+``__annotations__`` dictionary or directly creating a :class:`ForwardRef` object.

Doc/library/annotationlib.rst

@@ -83,7 +83,7 @@ The following functions are provided:
    Insert *x* in *a* in sorted order.
 
    This function first runs :py:func:`~bisect.bisect_left` to locate an insertion point.
-   Next, it runs the :meth:`!insert` method on *a* to insert *x* at the
+   Next, it runs the :meth:`~sequence.insert` method on *a* to insert *x* at the
    appropriate position to maintain sort order.
 
    To support inserting records in a table, the *key* function (if any) is
@@ -103,7 +103,7 @@ The following functions are provided:
    entries of *x*.
 
    This function first runs :py:func:`~bisect.bisect_right` to locate an insertion point.
-   Next, it runs the :meth:`!insert` method on *a* to insert *x* at the
+   Next, it runs the :meth:`~sequence.insert` method on *a* to insert *x* at the
    appropriate position to maintain sort order.
 
    To support inserting records in a table, the *key* function (if any) is

Doc/library/bisect.rst

@@ -264,8 +264,9 @@ Collections Abstract Base Classes -- Detailed Descriptions
    ABCs for read-only and mutable :term:`sequences <sequence>`.
 
    Implementation note: Some of the mixin methods, such as
-   :meth:`~container.__iter__`, :meth:`~object.__reversed__` and :meth:`index`, make
-   repeated calls to the underlying :meth:`~object.__getitem__` method.
+   :meth:`~container.__iter__`, :meth:`~object.__reversed__`,
+   and :meth:`~sequence.index` make repeated calls to the underlying
+   :meth:`~object.__getitem__` method.
    Consequently, if :meth:`~object.__getitem__` is implemented with constant
    access speed, the mixin methods will have linear performance;
    however, if the underlying method is linear (as it would be with a
@@ -281,8 +282,8 @@ Collections Abstract Base Classes -- Detailed Descriptions
       Supporting the *start* and *stop* arguments is optional, but recommended.
 
       .. versionchanged:: 3.5
-         The :meth:`!index` method added support for *stop* and *start*
-         arguments.
+         The :meth:`~sequence.index` method gained support for
+         the *stop* and *start* arguments.
 
 .. class:: Set
            MutableSet