[pull] main from python:main

.github/workflows/jit.yml

@@ -134,6 +134,34 @@ jobs:
           make all --jobs 4
           ./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
 
+  jit-with-disabled-gil:
+    name: Free-Threaded (Debug)
+    needs: interpreter
+    runs-on: ubuntu-24.04
+    timeout-minutes: 90
+    strategy:
+      fail-fast: false
+      matrix:
+        llvm:
+          - 19
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+      - name: Build with JIT enabled and GIL disabled
+        run: |
+          sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
+          export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
+          ./configure --enable-experimental-jit --with-pydebug --disable-gil
+          make all --jobs 4
+      - name: Run tests
+        run: |
+          ./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
+        continue-on-error: true
+
   no-opt-jit:
     name: JIT without optimizations (Debug)
     needs: interpreter
@@ -160,31 +188,3 @@ jobs:
       - name: Run tests without optimizations
         run: |
           PYTHON_UOPS_OPTIMIZE=0 ./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
-
-  # XXX: GH-133171
-  # jit-with-disabled-gil:
-  #   name: Free-Threaded (Debug)
-  #   needs: interpreter
-  #   runs-on: ubuntu-24.04
-  #   timeout-minutes: 90
-  #   strategy:
-  #     fail-fast: false
-  #     matrix:
-  #       llvm:
-  #         - 19
-  #   steps:
-  #     - uses: actions/checkout@v4
-  #       with:
-  #         persist-credentials: false
-  #     - uses: actions/setup-python@v5
-  #       with:
-  #         python-version: '3.11'
-  #     - name: Build with JIT enabled and GIL disabled
-  #       run: |
-  #         sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
-  #         export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
-  #         ./configure --enable-experimental-jit --with-pydebug --disable-gil
-  #         make all --jobs 4
-  #     - name: Run tests
-  #       run: |
-  #         ./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3

Doc/c-api/weakref.rst

@@ -64,30 +64,6 @@ as much as it can.
    .. versionadded:: 3.13
 
 
-.. c:function:: PyObject* PyWeakref_GetObject(PyObject *ref)
-
-   Return a :term:`borrowed reference` to the referenced object from a weak
-   reference, *ref*.  If the referent is no longer live, returns ``Py_None``.
-
-   .. note::
-
-      This function returns a :term:`borrowed reference` to the referenced object.
-      This means that you should always call :c:func:`Py_INCREF` on the object
-      except when it cannot be destroyed before the last usage of the borrowed
-      reference.
-
-   .. deprecated-removed:: 3.13 3.15
-      Use :c:func:`PyWeakref_GetRef` instead.
-
-
-.. c:function:: PyObject* PyWeakref_GET_OBJECT(PyObject *ref)
-
-   Similar to :c:func:`PyWeakref_GetObject`, but does no error checking.
-
-   .. deprecated-removed:: 3.13 3.15
-      Use :c:func:`PyWeakref_GetRef` instead.
-
-
 .. c:function:: int PyWeakref_IsDead(PyObject *ref)
 
    Test if the weak reference *ref* is dead. Returns 1 if the reference is

Doc/data/refcounts.dat

@@ -2947,12 +2947,6 @@ PyWeakref_CheckProxy:PyObject*:ob:0:
 PyWeakref_CheckRef:int:::
 PyWeakref_CheckRef:PyObject*:ob:0:
 
-PyWeakref_GET_OBJECT:PyObject*::0:
-PyWeakref_GET_OBJECT:PyObject*:ref:0:
-
-PyWeakref_GetObject:PyObject*::0:
-PyWeakref_GetObject:PyObject*:ref:0:
-
 PyWeakref_GetRef:int:::
 PyWeakref_GetRef:PyObject*:ref:0:
 PyWeakref_GetRef:PyObject**:pobj:+1:

Doc/deprecations/c-api-pending-removal-in-3.15.rst

@@ -3,7 +3,7 @@ Pending removal in Python 3.15
 
 * The :c:func:`!PyImport_ImportModuleNoBlock`:
   Use :c:func:`PyImport_ImportModule` instead.
-* :c:func:`PyWeakref_GetObject` and :c:func:`PyWeakref_GET_OBJECT`:
+* :c:func:`!PyWeakref_GetObject` and :c:func:`!PyWeakref_GET_OBJECT`:
   Use :c:func:`PyWeakref_GetRef` instead. The `pythoncapi-compat project
   <https://github.com/python/pythoncapi-compat/>`__ can be used to get
   :c:func:`PyWeakref_GetRef` on Python 3.12 and older.

Doc/howto/free-threading-extensions.rst

@@ -173,9 +173,9 @@ that return :term:`strong references <strong reference>`.
 +-----------------------------------+-----------------------------------+
 | :c:func:`PyDict_Next`             | none (see :ref:`PyDict_Next`)     |
 +-----------------------------------+-----------------------------------+
-| :c:func:`PyWeakref_GetObject`     | :c:func:`PyWeakref_GetRef`        |
+| :c:func:`!PyWeakref_GetObject`    | :c:func:`PyWeakref_GetRef`        |
 +-----------------------------------+-----------------------------------+
-| :c:func:`PyWeakref_GET_OBJECT`    | :c:func:`PyWeakref_GetRef`        |
+| :c:func:`!PyWeakref_GET_OBJECT`   | :c:func:`PyWeakref_GetRef`        |
 +-----------------------------------+-----------------------------------+
 | :c:func:`PyImport_AddModule`      | :c:func:`PyImport_AddModuleRef`   |
 +-----------------------------------+-----------------------------------+

Doc/library/codecs.rst

@@ -1339,7 +1339,7 @@ particular, the following variants typically exist:
 +-----------------+--------------------------------+--------------------------------+
 | utf_8           | U8, UTF, utf8, cp65001         | all languages                  |
 +-----------------+--------------------------------+--------------------------------+
-| utf_8_sig       |                                | all languages                  |
+| utf_8_sig       | utf8-sig                       | all languages                  |
 +-----------------+--------------------------------+--------------------------------+
 
 .. versionchanged:: 3.4

Doc/whatsnew/3.11.rst

@@ -2673,7 +2673,7 @@ Removed
 
   (Contributed by Victor Stinner in :issue:`45474`.)
 
-* Exclude :c:func:`PyWeakref_GET_OBJECT` from the limited C API. It never
+* Exclude :c:func:`!PyWeakref_GET_OBJECT` from the limited C API. It never
   worked since the :c:type:`!PyWeakReference` structure is opaque in the
   limited C API.
   (Contributed by Victor Stinner in :issue:`35134`.)

Doc/whatsnew/3.13.rst

@@ -2246,7 +2246,7 @@ New Features
   (Contributed by Serhiy Storchaka in :gh:`110289`.)
 
 * Add the :c:func:`PyWeakref_GetRef` function
-  as an alternative to  :c:func:`PyWeakref_GetObject`
+  as an alternative to  :c:func:`!PyWeakref_GetObject`
   that returns a :term:`strong reference`
   or ``NULL`` if the referent is no longer live.
   (Contributed by Victor Stinner in :gh:`105927`.)
@@ -2531,8 +2531,8 @@ Deprecated C APIs
   are just aliases to :c:type:`!wchar_t`.
   (Contributed by Victor Stinner in :gh:`105156`.)
 
-* Deprecate the :c:func:`PyWeakref_GetObject` and
-  :c:func:`PyWeakref_GET_OBJECT` functions,
+* Deprecate the :c:func:`!PyWeakref_GetObject` and
+  :c:func:`!PyWeakref_GET_OBJECT` functions,
   which return a :term:`borrowed reference`.
   Replace them with the new :c:func:`PyWeakref_GetRef` function,
   which returns a :term:`strong reference`.

Doc/whatsnew/3.15.rst

@@ -869,6 +869,11 @@ Removed C APIs
   of :c:func:`PyImport_ImportModule`.
   (Contributed by Bénédikt Tran in :gh:`133644`.)
 
+* :c:func:`!PyWeakref_GetObject` and :c:macro:`!PyWeakref_GET_OBJECT`:
+  use :c:func:`PyWeakref_GetRef` instead. The |pythoncapi_compat_project|
+  can be used to get :c:func:`!PyWeakref_GetRef` on Python 3.12 and older.
+  (Contributed by Bénédikt Tran in :gh:`133644`.)
+
 * Remove deprecated :c:func:`!PySys_ResetWarnOptions`.
   Clear :data:`sys.warnoptions` and :data:`!warnings.filters` instead.
 

Include/cpython/weakrefobject.h

@@ -47,20 +47,3 @@ PyAPI_FUNC(void) _PyWeakref_ClearRef(PyWeakReference *self);
 
 // Test if a weak reference is dead.
 PyAPI_FUNC(int) PyWeakref_IsDead(PyObject *ref);
-
-Py_DEPRECATED(3.13) static inline PyObject* PyWeakref_GET_OBJECT(PyObject *ref_obj)
-{
-    PyWeakReference *ref = _PyWeakref_CAST(ref_obj);
-    PyObject *obj = ref->wr_object;
-    // Explanation for the Py_REFCNT() check: when a weakref's target is part
-    // of a long chain of deallocations which triggers the trashcan mechanism,
-    // clearing the weakrefs can be delayed long after the target's refcount
-    // has dropped to zero.  In the meantime, code accessing the weakref will
-    // be able to "see" the target object even though it is supposed to be
-    // unreachable.  See issue gh-60806.
-    if (Py_REFCNT(obj) > 0) {
-        return obj;
-    }
-    return Py_None;
-}
-#define PyWeakref_GET_OBJECT(ref) PyWeakref_GET_OBJECT(_PyObject_CAST(ref))

Include/internal/pycore_stackref.h

@@ -464,6 +464,12 @@ PyStackRef_CLOSE_SPECIALIZED(_PyStackRef ref, destructor destruct)
     PyStackRef_CLOSE(ref);
 }
 
+static inline int
+PyStackRef_RefcountOnObject(_PyStackRef ref)
+{
+    return (ref.bits & Py_TAG_REFCNT) == 0;
+}
+
 static inline _PyStackRef
 PyStackRef_DUP(_PyStackRef stackref)
 {

Include/weakrefobject.h

@@ -27,7 +27,6 @@ PyAPI_FUNC(PyObject *) PyWeakref_NewRef(PyObject *ob,
                                         PyObject *callback);
 PyAPI_FUNC(PyObject *) PyWeakref_NewProxy(PyObject *ob,
                                           PyObject *callback);
-Py_DEPRECATED(3.13) PyAPI_FUNC(PyObject *) PyWeakref_GetObject(PyObject *ref);
 
 #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030D0000
 PyAPI_FUNC(int) PyWeakref_GetRef(PyObject *ref, PyObject **pobj);

Lib/encodings/aliases.py

@@ -17,7 +17,7 @@
 """
 aliases = {
 
-    # Please keep this list sorted alphabetically by value !
+    # Please keep this list sorted alphabetically by value!
 
     # ascii codec
     '646'                : 'ascii',
@@ -554,6 +554,9 @@
     'utf8_ucs4'          : 'utf_8',
     'cp65001'            : 'utf_8',
 
+    # utf_8_sig codec
+    'utf8_sig'           : 'utf_8_sig',
+
     # uu_codec codec
     'uu'                 : 'uu_codec',
 

Lib/pdb.py

@@ -100,7 +100,6 @@
 import _pyrepl.utils
 
 from contextlib import ExitStack, closing, contextmanager
-from rlcompleter import Completer
 from types import CodeType
 from warnings import deprecated
 
@@ -364,6 +363,15 @@ def __init__(self, completekey='tab', stdin=None, stdout=None, skip=None,
             readline.set_completer_delims(' \t\n`@#%^&*()=+[{]}\\|;:\'",<>?')
         except ImportError:
             pass
+
+        # GH-138860
+        # We need to lazy-import rlcompleter to avoid deadlock
+        # We cannot import it during self.complete* methods because importing
+        # rlcompleter for the first time will overwrite readline's completer
+        # So we import it here and save the Completer class
+        from rlcompleter import Completer
+        self.RlCompleter = Completer
+
         self.allow_kbdint = False
         self.nosigint = nosigint
         # Consider these characters as part of the command so when the users type
@@ -1186,10 +1194,9 @@ def completedefault(self, text, line, begidx, endidx):
             conv_vars = self.curframe.f_globals.get('__pdb_convenience_variables', {})
             return [f"${name}" for name in conv_vars if name.startswith(text[1:])]
 
-        # Use rlcompleter to do the completion
         state = 0
         matches = []
-        completer = Completer(self.curframe.f_globals | self.curframe.f_locals)
+        completer = self.RlCompleter(self.curframe.f_globals | self.curframe.f_locals)
         while (match := completer.complete(text, state)) is not None:
             matches.append(match)
             state += 1
@@ -1204,8 +1211,8 @@ def _enable_rlcompleter(self, ns):
             return
 
         try:
+            completer = self.RlCompleter(ns)
             old_completer = readline.get_completer()
-            completer = Completer(ns)
             readline.set_completer(completer.complete)
             yield
         finally:

Lib/test/test_pdb.py

@@ -4688,6 +4688,28 @@ def foo():
         stdout, _ = self._run_script(script, commands)
         self.assertIn("42", stdout)
 
+    def test_readline_not_imported(self):
+        """GH-138860
+        Directly or indirectly importing readline might deadlock a subprocess
+        if it's launched with process_group=0 or preexec_fn=setpgrp
+
+        It's also a pattern that readline is never imported with just import pdb.
+
+        This test is to ensure that readline is not imported for import pdb.
+        It's possible that we have a good reason to do that in the future.
+        """
+
+        script = textwrap.dedent("""
+            import sys
+            import pdb
+            if "readline" in sys.modules:
+                print("readline imported")
+        """)
+        commands = ""
+        stdout, stderr = self._run_script(script, commands)
+        self.assertNotIn("readline imported", stdout)
+        self.assertEqual(stderr, "")
+
 
 @support.force_colorized_test_class
 class PdbTestColorize(unittest.TestCase):

Misc/NEWS.d/3.11.0a2.rst

@@ -1188,7 +1188,7 @@ context objects can now be disabled.
 .. nonce: Z0Zk_m
 .. section: C API
 
-Exclude :c:func:`PyWeakref_GET_OBJECT` from the limited C API. It never
+Exclude :c:func:`!PyWeakref_GET_OBJECT` from the limited C API. It never
 worked since the :c:type:`!PyWeakReference` structure is opaque in the
 limited C API.
 

Misc/NEWS.d/3.13.0a1.rst

@@ -6458,8 +6458,8 @@ Victor Stinner.
 .. nonce: GRxZtI
 .. section: C API
 
-Deprecate the :c:func:`PyWeakref_GetObject` and
-:c:func:`PyWeakref_GET_OBJECT` functions: use the new
+Deprecate the :c:func:`!PyWeakref_GetObject` and
+:c:func:`!PyWeakref_GET_OBJECT` functions: use the new
 :c:func:`PyWeakref_GetRef` function instead. Patch by Victor Stinner.
 
 ..
@@ -6470,7 +6470,7 @@ Deprecate the :c:func:`PyWeakref_GetObject` and
 .. section: C API
 
 Add :c:func:`PyWeakref_GetRef` function: similar to
-:c:func:`PyWeakref_GetObject` but returns a :term:`strong reference`, or
+:c:func:`!PyWeakref_GetObject` but returns a :term:`strong reference`, or
 ``NULL`` if the referent is no longer live. Patch by Victor Stinner.
 
 ..

Misc/NEWS.d/next/C_API/2025-05-08-12-40-59.gh-issue-133644.FNexLJ.rst

@@ -0,0 +1,3 @@
+Remove deprecated function :c:func:`!PyWeakref_GetObject` and macro
+:c:macro:`!PyWeakref_GET_OBJECT`. Use :c:func:`PyWeakref_GetRef` instead.
+Patch by Bénédikt Tran.

Misc/NEWS.d/next/Library/2025-07-11-08-15-17.gh-issue-83336.ptpmq7.rst

@@ -0,0 +1 @@
+``utf8_sig`` is now aliased to :mod:`encodings.utf_8_sig`

Misc/NEWS.d/next/Library/2025-09-20-17-50-31.gh-issue-138860.Y9JXap.rst

@@ -0,0 +1 @@
+Lazy import :mod:`rlcompleter` in :mod:`pdb` to avoid deadlock in subprocess.

Misc/stable_abi.toml

@@ -1600,6 +1600,7 @@
     added = '3.2'
 [function.PyWeakref_GetObject]
     added = '3.2'
+    abi_only = true
 [function.PyWeakref_NewProxy]
     added = '3.2'
 [function.PyWeakref_NewRef]