Merge branch 'main' into htmlparser-cdata

Author: serhiy-storchaka

.github/workflows/jit.yml

@@ -117,6 +117,10 @@ jobs:
           find /usr/local/bin -lname '*/Library/Frameworks/Python.framework/*' -delete
           brew install llvm@${{ matrix.llvm }}
           export SDKROOT="$(xcrun --show-sdk-path)"
+          # Set MACOSX_DEPLOYMENT_TARGET and -Werror=unguarded-availability to
+          # make sure we don't break downstream distributors (like uv):
+          export CFLAGS_JIT='-Werror=unguarded-availability'
+          export MACOSX_DEPLOYMENT_TARGET=10.15
           ./configure --enable-experimental-jit --enable-universalsdk --with-universal-archs=universal2 ${{ matrix.debug && '--with-pydebug' || '' }}
           make all --jobs 4
           ./python.exe -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3

Doc/c-api/complex.rst

@@ -43,24 +43,36 @@ pointers.  This is consistent throughout the API.
    Return the sum of two complex numbers, using the C :c:type:`Py_complex`
    representation.
 
+   .. deprecated:: 3.15
+      This function is :term:`soft deprecated`.
+
 
 .. c:function:: Py_complex _Py_c_diff(Py_complex left, Py_complex right)
 
    Return the difference between two complex numbers, using the C
    :c:type:`Py_complex` representation.
 
+   .. deprecated:: 3.15
+      This function is :term:`soft deprecated`.
+
 
 .. c:function:: Py_complex _Py_c_neg(Py_complex num)
 
    Return the negation of the complex number *num*, using the C
    :c:type:`Py_complex` representation.
 
+   .. deprecated:: 3.15
+      This function is :term:`soft deprecated`.
+
 
 .. c:function:: Py_complex _Py_c_prod(Py_complex left, Py_complex right)
 
    Return the product of two complex numbers, using the C :c:type:`Py_complex`
    representation.
 
+   .. deprecated:: 3.15
+      This function is :term:`soft deprecated`.
+
 
 .. c:function:: Py_complex _Py_c_quot(Py_complex dividend, Py_complex divisor)
 
@@ -70,6 +82,9 @@ pointers.  This is consistent throughout the API.
    If *divisor* is null, this method returns zero and sets
    :c:data:`errno` to :c:macro:`!EDOM`.
 
+   .. deprecated:: 3.15
+      This function is :term:`soft deprecated`.
+
 
 .. c:function:: Py_complex _Py_c_pow(Py_complex num, Py_complex exp)
 
@@ -81,6 +96,19 @@ pointers.  This is consistent throughout the API.
 
    Set :c:data:`errno` to :c:macro:`!ERANGE` on overflows.
 
+   .. deprecated:: 3.15
+      This function is :term:`soft deprecated`.
+
+
+.. c:function:: double _Py_c_abs(Py_complex num)
+
+   Return the absolute value of the complex number *num*.
+
+   Set :c:data:`errno` to :c:macro:`!ERANGE` on overflows.
+
+   .. deprecated:: 3.15
+      This function is :term:`soft deprecated`.
+
 
 Complex Numbers as Python Objects
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Doc/c-api/init.rst

@@ -2287,7 +2287,7 @@ The C-API provides a basic mutual exclusion lock.
       should not be used to make concurrency control decisions, as the lock
       state may change immediately after the check.
 
-   .. versionadded:: next
+   .. versionadded:: 3.14
 
 .. _python-critical-section-api:
 
@@ -2372,7 +2372,7 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
 
    On the default build, this macro expands to ``{``.
 
-   .. versionadded:: next
+   .. versionadded:: 3.14
 
 .. c:macro:: Py_END_CRITICAL_SECTION()
 
@@ -2418,7 +2418,7 @@ code triggered by the finalizer blocks and calls :c:func:`PyEval_SaveThread`.
 
    On the default build, this macro expands to ``{``.
 
-   .. versionadded:: next
+   .. versionadded:: 3.14
 
 .. c:macro:: Py_END_CRITICAL_SECTION2()
 

Doc/c-api/perfmaps.rst

@@ -5,11 +5,12 @@
 Support for Perf Maps
 ----------------------
 
-On supported platforms (as of this writing, only Linux), the runtime can take
+On supported platforms (Linux and macOS), the runtime can take
 advantage of *perf map files* to make Python functions visible to an external
-profiling tool (such as `perf <https://perf.wiki.kernel.org/index.php/Main_Page>`_).
-A running process may create a file in the ``/tmp`` directory, which contains entries
-that can map a section of executable code to a name. This interface is described in the
+profiling tool (such as `perf <https://perf.wiki.kernel.org/index.php/Main_Page>`_ or
+`samply <https://github.com/mstange/samply/>`_). A running process may create a
+file in the ``/tmp`` directory, which contains entries that can map a section
+of executable code to a name. This interface is described in the
 `documentation of the Linux Perf tool <https://git.kernel.org/pub/scm/linux/
 kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/jit-interface.txt>`_.
 

Doc/extending/extending.rst

@@ -214,7 +214,7 @@ and initialize it by calling :c:func:`PyErr_NewException` in the module's
 
    SpamError = PyErr_NewException("spam.error", NULL, NULL);
 
-Since :c:data:`!SpamError` is a global variable, it will be overwitten every time
+Since :c:data:`!SpamError` is a global variable, it will be overwritten every time
 the module is reinitialized, when the :c:data:`Py_mod_exec` function is called.
 
 For now, let's avoid the issue: we will block repeated initialization by raising an

Doc/howto/free-threading-extensions.rst

@@ -161,6 +161,8 @@ that return :term:`strong references <strong reference>`.
 +===================================+===================================+
 | :c:func:`PyList_GetItem`          | :c:func:`PyList_GetItemRef`       |
 +-----------------------------------+-----------------------------------+
+| :c:func:`PyList_GET_ITEM`         | :c:func:`PyList_GetItemRef`       |
++-----------------------------------+-----------------------------------+
 | :c:func:`PyDict_GetItem`          | :c:func:`PyDict_GetItemRef`       |
 +-----------------------------------+-----------------------------------+
 | :c:func:`PyDict_GetItemWithError` | :c:func:`PyDict_GetItemRef`       |

Doc/howto/perf_profiling.rst

@@ -2,34 +2,35 @@
 
 .. _perf_profiling:
 
-==============================================
-Python support for the Linux ``perf`` profiler
-==============================================
+========================================================
+Python support for the ``perf map`` compatible profilers
+========================================================
 
 :author: Pablo Galindo
 
-`The Linux perf profiler <https://perf.wiki.kernel.org>`_
-is a very powerful tool that allows you to profile and obtain
-information about the performance of your application.
-``perf`` also has a very vibrant ecosystem of tools
-that aid with the analysis of the data that it produces.
+`The Linux perf profiler <https://perf.wiki.kernel.org>`_ and
+`samply <https://github.com/mstange/samply>`_ are powerful tools that allow you to
+profile and obtain information about the performance of your application.
+Both tools have vibrant ecosystems that aid with the analysis of the data they produce.
 
-The main problem with using the ``perf`` profiler with Python applications is that
-``perf`` only gets information about native symbols, that is, the names of
+The main problem with using these profilers with Python applications is that
+they only get information about native symbols, that is, the names of
 functions and procedures written in C. This means that the names and file names
-of Python functions in your code will not appear in the output of ``perf``.
+of Python functions in your code will not appear in the profiler output.
 
 Since Python 3.12, the interpreter can run in a special mode that allows Python
-functions to appear in the output of the ``perf`` profiler. When this mode is
+functions to appear in the output of compatible profilers. When this mode is
 enabled, the interpreter will interpose a small piece of code compiled on the
-fly before the execution of every Python function and it will teach ``perf`` the
+fly before the execution of every Python function and it will teach the profiler the
 relationship between this piece of code and the associated Python function using
 :doc:`perf map files <../c-api/perfmaps>`.
 
 .. note::
 
-    Support for the ``perf`` profiler is currently only available for Linux on
-    select architectures. Check the output of the ``configure`` build step or
+    Support for profiling is available on Linux and macOS on select architectures.
+    Perf is available on Linux, while samply can be used on both Linux and macOS.
+    samply support on macOS is available starting from Python 3.15.
+    Check the output of the ``configure`` build step or
     check the output of ``python -m sysconfig | grep HAVE_PERF_TRAMPOLINE``
     to see if your system is supported.
 
@@ -148,6 +149,31 @@ Instead, if we run the same experiment with ``perf`` support enabled we get:
 
 
 
+Using the samply profiler
+-------------------------
+
+samply is a modern profiler that can be used as an alternative to perf.
+It uses the same perf map files that Python generates, making it compatible
+with Python's profiling support. samply is particularly useful on macOS
+where perf is not available.
+
+To use samply with Python, first install it following the instructions at
+https://github.com/mstange/samply, then run::
+
+    $ samply record PYTHONPERFSUPPORT=1 python my_script.py
+
+This will open a web interface where you can analyze the profiling data
+interactively. The advantage of samply is that it provides a modern
+web-based interface for analyzing profiling data and works on both Linux
+and macOS.
+
+On macOS, samply support requires Python 3.15 or later. Also on macOS, samply
+can't profile signed Python executables due to restrictions by macOS. You can
+profile with Python binaries that you've compiled yourself, or which are
+unsigned or locally-signed (such as anything installed by Homebrew). In
+order to attach to running processes on macOS, run ``samply setup`` once (and
+every time samply is updated) to self-sign the samply binary.
+
 How to enable ``perf`` profiling support
 ----------------------------------------
 

Doc/library/array.rst

@@ -24,7 +24,7 @@ defined:
 +-----------+--------------------+-------------------+-----------------------+-------+
 | ``'u'``   | wchar_t            | Unicode character | 2                     | \(1)  |
 +-----------+--------------------+-------------------+-----------------------+-------+
-| ``'w'``   | Py_UCS4            | Unicode character | 4                     |       |
+| ``'w'``   | Py_UCS4            | Unicode character | 4                     | \(2)  |
 +-----------+--------------------+-------------------+-----------------------+-------+
 | ``'h'``   | signed short       | int               | 2                     |       |
 +-----------+--------------------+-------------------+-----------------------+-------+
@@ -60,6 +60,9 @@ Notes:
    .. deprecated-removed:: 3.3 3.16
       Please migrate to ``'w'`` typecode.
 
+(2)
+   .. versionadded:: 3.13
+
 
 The actual representation of values is determined by the machine architecture
 (strictly speaking, by the C implementation).  The actual size can be accessed

Doc/library/asyncio-eventloop.rst

@@ -611,6 +611,12 @@ Opening network connections
      to bind the socket locally.  The *local_host* and *local_port*
      are looked up using :meth:`getaddrinfo`.
 
+     .. note::
+
+        On Windows, when using the proactor event loop with ``local_addr=None``,
+        an :exc:`OSError` with :attr:`!errno.WSAEINVAL` will be raised
+        when running it.
+
    * *remote_addr*, if given, is a ``(remote_host, remote_port)`` tuple used
      to connect the socket to a remote address.  The *remote_host* and
      *remote_port* are looked up using :meth:`getaddrinfo`.

Doc/library/asyncio-queue.rst

@@ -102,17 +102,34 @@ Queue
 
    .. method:: shutdown(immediate=False)
 
-      Shut down the queue, making :meth:`~Queue.get` and :meth:`~Queue.put`
+      Put a :class:`Queue` instance into a shutdown mode.
+
+      The queue can no longer grow.
+      Future calls to :meth:`~Queue.put` raise :exc:`QueueShutDown`.
+      Currently blocked callers of :meth:`~Queue.put` will be unblocked
+      and will raise :exc:`QueueShutDown` in the formerly blocked thread.
+
+      If *immediate* is false (the default), the queue can be wound
+      down normally with :meth:`~Queue.get` calls to extract tasks
+      that have already been loaded.
+
+      And if :meth:`~Queue.task_done` is called for each remaining task, a
+      pending :meth:`~Queue.join` will be unblocked normally.
+
+      Once the queue is empty, future calls to :meth:`~Queue.get` will
       raise :exc:`QueueShutDown`.
 
-      By default, :meth:`~Queue.get` on a shut down queue will only
-      raise once the queue is empty. Set *immediate* to true to make
-      :meth:`~Queue.get` raise immediately instead.
+      If *immediate* is true, the queue is terminated immediately.
+      The queue is drained to be completely empty and the count
+      of unfinished tasks is reduced by the number of tasks drained.
+      If unfinished tasks is zero, callers of :meth:`~Queue.join`
+      are unblocked.  Also, blocked callers of :meth:`~Queue.get`
+      are unblocked and will raise :exc:`QueueShutDown` because the
+      queue is empty.
 
-      All blocked callers of :meth:`~Queue.put` and :meth:`~Queue.get`
-      will be unblocked. If *immediate* is true, a task will be marked
-      as done for each remaining item in the queue, which may unblock
-      callers of :meth:`~Queue.join`.
+      Use caution when using :meth:`~Queue.join` with *immediate* set
+      to true. This unblocks the join even when no work has been done
+      on the tasks, violating the usual invariant for joining a queue.
 
       .. versionadded:: 3.13
 
@@ -129,9 +146,6 @@ Queue
       call was received for every item that had been :meth:`~Queue.put`
       into the queue).
 
-      ``shutdown(immediate=True)`` calls :meth:`task_done` for each
-      remaining item in the queue.
-
       Raises :exc:`ValueError` if called more times than there were
       items placed in the queue.