Merge branch 'main' into check-mro-lookup-not-null

Author: emmatyping

.devcontainer/devcontainer.json

@@ -1,5 +1,5 @@
 {
-    "image": "ghcr.io/python/devcontainer:2024.09.25.11038928730",
+    "image": "ghcr.io/python/devcontainer:2025.05.25.15232270922",
     "onCreateCommand": [
         // Install common tooling.
         "dnf",

.gitignore

@@ -171,5 +171,10 @@ Python/frozen_modules/MANIFEST
 /python
 !/Python/
 
+# People's custom https://docs.anthropic.com/en/docs/claude-code/memory configs.
+/.claude/
+CLAUDE.local.md
+
+#### main branch only stuff below this line, things to backport go above. ####
 # main branch only: ABI files are not checked/maintained.
 Doc/data/python*.abi

Doc/c-api/init_config.rst

@@ -2111,7 +2111,7 @@ initialization::
 
         /* Specify sys.path explicitly */
         /* If you want to modify the default set of paths, finish
-           initialization first and then use PySys_GetObject("path") */
+           initialization first and then use PySys_GetAttrString("path") */
         config.module_search_paths_set = 1;
         status = PyWideStringList_Append(&config.module_search_paths,
                                          L"/path/to/stdlib");

Doc/c-api/intro.rst

@@ -838,3 +838,41 @@ after every statement run by the interpreter.)
 
 Please refer to :file:`Misc/SpecialBuilds.txt` in the Python source distribution
 for more detailed information.
+
+
+.. _c-api-tools:
+
+Recommended third party tools
+=============================
+
+The following third party tools offer both simpler and more sophisticated
+approaches to creating C, C++ and Rust extensions for Python:
+
+* `Cython <https://cython.org/>`_
+* `cffi <https://cffi.readthedocs.io>`_
+* `HPy <https://hpyproject.org/>`_
+* `nanobind <https://github.com/wjakob/nanobind>`_ (C++)
+* `Numba <https://numba.pydata.org/>`_
+* `pybind11 <https://pybind11.readthedocs.io/>`_ (C++)
+* `PyO3 <https://pyo3.rs/>`_ (Rust)
+* `SWIG <https://www.swig.org>`_
+
+Using tools such as these can help avoid writing code that is tightly bound to
+a particular version of CPython, avoid reference counting errors, and focus
+more on your own code than on using the CPython API. In general, new versions
+of Python can be supported by updating the tool, and your code will often use
+newer and more efficient APIs automatically. Some tools also support compiling
+for other implementations of Python from a single set of sources.
+
+These projects are not supported by the same people who maintain Python, and
+issues need to be raised with the projects directly. Remember to check that the
+project is still maintained and supported, as the list above may become
+outdated.
+
+.. seealso::
+
+   `Python Packaging User Guide: Binary Extensions <https://packaging.python.org/guides/packaging-binary-extensions/>`_
+      The Python Packaging User Guide not only covers several available
+      tools that simplify the creation of binary extensions, but also
+      discusses the various reasons why creating an extension module may be
+      desirable in the first place.

Doc/c-api/module.rst

@@ -288,22 +288,40 @@ An alternate way to specify extensions is to request "multi-phase initialization
 Extension modules created this way behave more like Python modules: the
 initialization is split between the *creation phase*, when the module object
 is created, and the *execution phase*, when it is populated.
-The distinction is similar to the :py:meth:`!__new__` and :py:meth:`!__init__` methods
-of classes.
+The distinction is similar to the :py:meth:`~object.__new__` and
+:py:meth:`~object.__init__` methods of classes.
 
 Unlike modules created using single-phase initialization, these modules are not
-singletons: if the *sys.modules* entry is removed and the module is re-imported,
-a new module object is created, and the old module is subject to normal garbage
-collection -- as with Python modules.
-By default, multiple modules created from the same definition should be
-independent: changes to one should not affect the others.
-This means that all state should be specific to the module object (using e.g.
-using :c:func:`PyModule_GetState`), or its contents (such as the module's
-:attr:`~object.__dict__` or individual classes created with :c:func:`PyType_FromSpec`).
+singletons.
+For example, if the :py:attr:`sys.modules` entry is removed and the module
+is re-imported, a new module object is created, and typically populated with
+fresh method and type objects.
+The old module is subject to normal garbage collection.
+This mirrors the behavior of pure-Python modules.
+
+Additional module instances may be created in
+:ref:`sub-interpreters <sub-interpreter-support>`
+or after after Python runtime reinitialization
+(:c:func:`Py_Finalize` and :c:func:`Py_Initialize`).
+In these cases, sharing Python objects between module instances would likely
+cause crashes or undefined behavior.
+
+To avoid such issues, each instance of an extension module should
+be *isolated*: changes to one instance should not implicitly affect the others,
+and all state, including references to Python objects, should be specific to
+a particular module instance.
+See :ref:`isolating-extensions-howto` for more details and a practical guide.
+
+A simpler way to avoid these issues is
+:ref:`raising an error on repeated initialization <isolating-extensions-optout>`.
 
 All modules created using multi-phase initialization are expected to support
-:ref:`sub-interpreters <sub-interpreter-support>`. Making sure multiple modules
-are independent is typically enough to achieve this.
+:ref:`sub-interpreters <sub-interpreter-support>`, or otherwise explicitly
+signal a lack of support.
+This is usually achieved by isolation or blocking repeated initialization,
+as above.
+A module may also be limited to the main interpreter using
+the :c:data:`Py_mod_multiple_interpreters` slot.
 
 To request multi-phase initialization, the initialization function
 (PyInit_modulename) returns a :c:type:`PyModuleDef` instance with non-empty

Doc/c-api/sys.rst

@@ -258,10 +258,57 @@ These are utility functions that make functionality from the :mod:`sys` module
 accessible to C code.  They all work with the current interpreter thread's
 :mod:`sys` module's dict, which is contained in the internal thread state structure.
 
+.. c:function:: PyObject *PySys_GetAttr(PyObject *name)
+
+   Get the attribute *name* of the :mod:`sys` module.
+   Return a :term:`strong reference`.
+   Raise :exc:`RuntimeError` and return ``NULL`` if it does not exist or
+   if the :mod:`sys` module cannot be found.
+
+   If the non-existing object should not be treated as a failure, you can use
+   :c:func:`PySys_GetOptionalAttr` instead.
+
+   .. versionadded:: next
+
+.. c:function:: PyObject *PySys_GetAttrString(const char *name)
+
+   This is the same as :c:func:`PySys_GetAttr`, but *name* is
+   specified as a :c:expr:`const char*` UTF-8 encoded bytes string,
+   rather than a :c:expr:`PyObject*`.
+
+   If the non-existing object should not be treated as a failure, you can use
+   :c:func:`PySys_GetOptionalAttrString` instead.
+
+   .. versionadded:: next
+
+.. c:function:: int PySys_GetOptionalAttr(PyObject *name, PyObject **result)
+
+   Variant of :c:func:`PySys_GetAttr` which doesn't raise
+   exception if the object does not exist.
+
+   * Set *\*result* to a new :term:`strong reference` to the object and
+     return ``1`` if the object exists.
+   * Set *\*result* to ``NULL`` and return ``0`` without setting an exception
+     if the object does not exist.
+   * Set an exception, set *\*result* to ``NULL``, and return ``-1``,
+     if an error occurred.
+
+   .. versionadded:: next
+
+.. c:function:: int PySys_GetOptionalAttrString(const char *name, PyObject **result)
+
+   This is the same as :c:func:`PySys_GetOptionalAttr`, but *name* is
+   specified as a :c:expr:`const char*` UTF-8 encoded bytes string,
+   rather than a :c:expr:`PyObject*`.
+
+   .. versionadded:: next
+
 .. c:function:: PyObject *PySys_GetObject(const char *name)
 
-   Return the object *name* from the :mod:`sys` module or ``NULL`` if it does
-   not exist, without setting an exception.
+   Similar to :c:func:`PySys_GetAttrString`, but return a :term:`borrowed
+   reference` and return ``NULL`` *without* setting exception on failure.
+
+   Preserves exception that was set before the call.
 
 .. c:function:: int PySys_SetObject(const char *name, PyObject *v)
 

Doc/c-api/unicode.rst

@@ -1461,10 +1461,6 @@ the user settings on the machine running the codec.
    .. versionadded:: 3.3
 
 
-Methods & Slots
-"""""""""""""""
-
-
 .. _unicodemethodsandslots:
 
 Methods and Slot Functions
@@ -1726,10 +1722,6 @@ They all return ``NULL`` or ``-1`` if an exception occurs.
    from user input, prefer calling :c:func:`PyUnicode_FromString` and
    :c:func:`PyUnicode_InternInPlace` directly.
 
-   .. impl-detail::
-
-      Strings interned this way are made :term:`immortal`.
-
 
 .. c:function:: unsigned int PyUnicode_CHECK_INTERNED(PyObject *str)
 
@@ -1806,9 +1798,24 @@ object.
 
    See also :c:func:`PyUnicodeWriter_DecodeUTF8Stateful`.
 
+.. c:function:: int PyUnicodeWriter_WriteASCII(PyUnicodeWriter *writer, const char *str, Py_ssize_t size)
+
+   Write the ASCII string *str* into *writer*.
+
+   *size* is the string length in bytes. If *size* is equal to ``-1``, call
+   ``strlen(str)`` to get the string length.
+
+   *str* must only contain ASCII characters. The behavior is undefined if
+   *str* contains non-ASCII characters.
+
+   On success, return ``0``.
+   On error, set an exception, leave the writer unchanged, and return ``-1``.
+
+   .. versionadded:: next
+
 .. c:function:: int PyUnicodeWriter_WriteWideChar(PyUnicodeWriter *writer, const wchar_t *str, Py_ssize_t size)
 
-   Writer the wide string *str* into *writer*.
+   Write the wide string *str* into *writer*.
 
    *size* is a number of wide characters. If *size* is equal to ``-1``, call
    ``wcslen(str)`` to get the string length.

Doc/data/stable_abi.dat

@@ -204,17 +204,32 @@ value must be in a particular range or must satisfy other conditions,
 :c:data:`PyExc_ValueError` is appropriate.
 
 You can also define a new exception that is unique to your module.
-For this, you can declare a static global object variable at the beginning
-of the file::
+The simplest way to do this is to declare a static global object variable at
+the beginning of the file::
 
-   static PyObject *SpamError;
+   static PyObject *SpamError = NULL;
 
-and initialize it with an exception object in the module's
+and initialize it by calling :c:func:`PyErr_NewException` in the module's
 :c:data:`Py_mod_exec` function (:c:func:`!spam_module_exec`)::
 
+   SpamError = PyErr_NewException("spam.error", NULL, NULL);
+
+Since :c:data:`!SpamError` is a global variable, it will be overwitten every time
+the module is reinitialized, when the :c:data:`Py_mod_exec` function is called.
+
+For now, let's avoid the issue: we will block repeated initialization by raising an
+:py:exc:`ImportError`::
+
+   static PyObject *SpamError = NULL;
+
    static int
    spam_module_exec(PyObject *m)
    {
+       if (SpamError != NULL) {
+           PyErr_SetString(PyExc_ImportError,
+                           "cannot initialize spam module more than once");
+           return -1;
+       }
        SpamError = PyErr_NewException("spam.error", NULL, NULL);
        if (PyModule_AddObjectRef(m, "SpamError", SpamError) < 0) {
            return -1;
@@ -253,6 +268,11 @@ needed to ensure that it will not be discarded, causing :c:data:`!SpamError` to
 become a dangling pointer. Should it become a dangling pointer, C code which
 raises the exception could cause a core dump or other unintended side effects.
 
+For now, the :c:func:`Py_DECREF` call to remove this reference is missing.
+Even when the Python interpreter shuts down, the global :c:data:`!SpamError`
+variable will not be garbage-collected. It will "leak".
+We did, however, ensure that this will happen at most once per process.
+
 We discuss the use of :c:macro:`PyMODINIT_FUNC` as a function return type later in this
 sample.
 

Doc/extending/extending.rst

@@ -26,19 +26,9 @@ Recommended third party tools
 =============================
 
 This guide only covers the basic tools for creating extensions provided
-as part of this version of CPython. Third party tools like
-`Cython <https://cython.org/>`_, `cffi <https://cffi.readthedocs.io>`_,
-`SWIG <https://www.swig.org>`_ and `Numba <https://numba.pydata.org/>`_
-offer both simpler and more sophisticated approaches to creating C and C++
-extensions for Python.
-
-.. seealso::
-
-   `Python Packaging User Guide: Binary Extensions <https://packaging.python.org/guides/packaging-binary-extensions/>`_
-      The Python Packaging User Guide not only covers several available
-      tools that simplify the creation of binary extensions, but also
-      discusses the various reasons why creating an extension module may be
-      desirable in the first place.
+as part of this version of CPython. Some :ref:`third party tools
+<c-api-tools>` offer both simpler and more sophisticated approaches to creating
+C and C++ extensions for Python.
 
 
 Creating extensions without third party tools