Merge remote-tracking branch 'upstream/main' into binary_subscr_to_op

Author: iritkatriel

Doc/using/configure.rst

@@ -618,6 +618,16 @@ also be used to improve performance.
    Enable computed gotos in evaluation loop (enabled by default on supported
    compilers).
 
+.. option:: --with-tail-call-interp
+
+   Enable interpreters using tail calls in CPython. If enabled, enabling PGO
+   (:option:`--enable-optimizations`) is highly recommended. This option specifically
+   requires a C compiler with proper tail call support, and the
+   `preserve_none <https://clang.llvm.org/docs/AttributeReference.html#preserve-none>`_
+   calling convention. For example, Clang 19 and newer supports this feature.
+
+   .. versionadded:: next
+
 .. option:: --without-mimalloc
 
    Disable the fast :ref:`mimalloc <mimalloc>` allocator

Doc/whatsnew/3.14.rst

@@ -68,6 +68,7 @@ Summary -- release highlights
 * :ref:`PEP 649: deferred evaluation of annotations <whatsnew314-pep649>`
 * :ref:`PEP 741: Python Configuration C API <whatsnew314-pep741>`
 * :ref:`PEP 761: Discontinuation of PGP signatures <whatsnew314-pep761>`
+* :ref:`A new tail-calling interpreter  <whatsnew314-tail-call>`
 
 
 New features
@@ -208,6 +209,37 @@ configuration mechanisms).
 .. seealso::
    :pep:`741`.
 
+.. _whatsnew314-tail-call:
+
+A new tail-calling interpreter
+------------------------------
+
+A new type of interpreter based on tail calls has been added to CPython.
+For certain newer compilers, this interpreter provides
+significantly better performance. Preliminary numbers on our machines suggest
+anywhere from -3% to 30% faster Python code, and a geometric mean of 9-15%
+faster on ``pyperformance`` depending on platform and architecture.
+
+This interpreter currently only works with Clang 19 and newer
+on x86-64 and AArch64 architectures. However, we expect
+that a future release of GCC will support this as well.
+
+This feature is opt-in for now. We highly recommend enabling profile-guided
+optimization with the new interpreter as it is the only configuration we have
+tested and can validate its improved performance.
+For further information on how to build Python, see
+:option:`--with-tail-call-interp`.
+
+.. note::
+
+   This is not to be confused with `tail call optimization`__ of Python
+   functions, which is currently not implemented in CPython.
+
+   __ https://en.wikipedia.org/wiki/Tail_call
+
+(Contributed by Ken Jin in :gh:`128718`, with ideas on how to implement this
+in CPython by Mark Shannon, Garret Gu, Haoran Xu, and Josh Haberman.)
+
 
 Other language changes
 ======================

Lib/test/test_asyncio/test_tasks.py

@@ -2757,7 +2757,6 @@ async def func():
     # Add patched Task & Future back to the test case
     cls.Task = Task
     cls.Future = Future
-    cls.all_tasks = tasks.all_tasks
 
     # Add an extra unit-test
     cls.test_subclasses_ctask_cfuture = test_subclasses_ctask_cfuture
@@ -2883,7 +2882,7 @@ class PyTask_CFutureSubclass_Tests(BaseTaskTests, test_utils.TestCase):
 
     Future = getattr(futures, '_CFuture', None)
     Task = tasks._PyTask
-    all_tasks = tasks._py_all_tasks
+    all_tasks = staticmethod(tasks._py_all_tasks)
 
 
 @unittest.skipUnless(hasattr(tasks, '_CTask'),
@@ -2916,7 +2915,7 @@ class PyTask_PyFuture_Tests(BaseTaskTests, SetMethodsTest,
 class PyTask_PyFuture_SubclassTests(BaseTaskTests, test_utils.TestCase):
     Task = tasks._PyTask
     Future = futures._PyFuture
-
+    all_tasks = staticmethod(tasks._py_all_tasks)
 
 @unittest.skipUnless(hasattr(tasks, '_CTask'),
                      'requires the C _asyncio module')