Merge branch 'main' into SimpleHTTPRequestHandler-query-and-fragment

Author: serhiy-storchaka

Doc/c-api/init.rst

@@ -919,8 +919,36 @@ Note that the ``PyGILState_*`` functions assume there is only one global
 interpreter (created automatically by :c:func:`Py_Initialize`).  Python
 supports the creation of additional interpreters (using
 :c:func:`Py_NewInterpreter`), but mixing multiple interpreters and the
-``PyGILState_*`` API is unsupported.
+``PyGILState_*`` API is unsupported. This is because :c:func:`PyGILState_Ensure`
+and similar functions default to :term:`attaching <attached thread state>` a
+:term:`thread state` for the main interpreter, meaning that the thread can't safely
+interact with the calling subinterpreter.
+
+Supporting subinterpreters in non-Python threads
+------------------------------------------------
+
+If you would like to support subinterpreters with non-Python created threads, you
+must use the ``PyThreadState_*`` API instead of the traditional ``PyGILState_*``
+API.
+
+In particular, you must store the interpreter state from the calling
+function and pass it to :c:func:`PyThreadState_New`, which will ensure that
+the :term:`thread state` is targeting the correct interpreter::
+
+   /* The return value of PyInterpreterState_Get() from the
+      function that created this thread. */
+   PyInterpreterState *interp = ThreadData->interp;
+   PyThreadState *tstate = PyThreadState_New(interp);
+   PyThreadState_Swap(tstate);
+
+   /* GIL of the subinterpreter is now held.
+      Perform Python actions here. */
+   result = CallSomeFunction();
+   /* evaluate result or handle exception */
 
+   /* Destroy the thread state. No Python API allowed beyond this point. */
+   PyThreadState_Clear(tstate);
+   PyThreadState_DeleteCurrent();
 
 .. _fork-and-threads:
 
@@ -1097,6 +1125,10 @@ code, or when embedding the Python interpreter:
    .. seealso:
       :c:func:`PyEval_ReleaseThread`
 
+   .. note::
+      Similar to :c:func:`PyGILState_Ensure`, this function will hang the
+      thread if the runtime is finalizing.
+
 
 The following functions use thread-local storage, and are not compatible
 with sub-interpreters:
@@ -1123,10 +1155,10 @@ with sub-interpreters:
    When the function returns, there will be an :term:`attached thread state`
    and the thread will be able to call arbitrary Python code.  Failure is a fatal error.
 
-   .. note::
-      Calling this function from a thread when the runtime is finalizing will
-      hang the thread until the program exits, even if the thread was not
-      created by Python.  Refer to
+   .. warning::
+      Calling this function when the runtime is finalizing is unsafe. Doing
+      so will either hang the thread until the program ends, or fully crash
+      the interpreter in rare cases. Refer to
       :ref:`cautions-regarding-runtime-finalization` for more details.
 
    .. versionchanged:: 3.14
@@ -1143,28 +1175,37 @@ with sub-interpreters:
    Every call to :c:func:`PyGILState_Ensure` must be matched by a call to
    :c:func:`PyGILState_Release` on the same thread.
 
-
 .. c:function:: PyThreadState* PyGILState_GetThisThreadState()
 
    Get the :term:`attached thread state` for this thread.  May return ``NULL`` if no
    GILState API has been used on the current thread.  Note that the main thread
    always has such a thread-state, even if no auto-thread-state call has been
    made on the main thread.  This is mainly a helper/diagnostic function.
 
-   .. seealso: :c:func:`PyThreadState_Get``
+   .. note::
+      This function does not account for :term:`thread states <thread state>` created
+      by something other than :c:func:`PyGILState_Ensure` (such as :c:func:`PyThreadState_New`).
+      Prefer :c:func:`PyThreadState_Get` or :c:func:`PyThreadState_GetUnchecked`
+      for most cases.
 
+   .. seealso: :c:func:`PyThreadState_Get``
 
 .. c:function:: int PyGILState_Check()
 
    Return ``1`` if the current thread is holding the :term:`GIL` and ``0`` otherwise.
    This function can be called from any thread at any time.
-   Only if it has had its Python thread state initialized and currently is
-   holding the :term:`GIL` will it return ``1``.
+   Only if it has had its :term:`thread state <attached thread state>` initialized
+   via :c:func:`PyGILState_Ensure` will it return ``1``.
    This is mainly a helper/diagnostic function.  It can be useful
    for example in callback contexts or memory allocation functions when
    knowing that the :term:`GIL` is locked can allow the caller to perform sensitive
    actions or otherwise behave differently.
 
+   .. note::
+      If the current Python process has ever created a subinterpreter, this
+      function will *always* return ``1``. Prefer :c:func:`PyThreadState_GetUnchecked`
+      for most cases.
+
    .. versionadded:: 3.4
 
 

Doc/deprecations/pending-removal-in-3.15.rst

@@ -20,7 +20,7 @@ Pending removal in Python 3.15
 
 * :mod:`http.server`:
 
-  * The obsolete and rarely used :class:`~http.server.CGIHTTPRequestHandler`
+  * The obsolete and rarely used :class:`!CGIHTTPRequestHandler`
     has been deprecated since Python 3.13.
     No direct replacement exists.
     *Anything* is better than CGI to interface

Doc/library/dataclasses.rst

@@ -304,9 +304,9 @@ Module contents
 
     .. versionadded:: 3.10
 
-   - ``doc``: optional docstring for this field.
+   - *doc*: optional docstring for this field.
 
-    .. versionadded:: 3.13
+    .. versionadded:: 3.14
 
    If the default value of a field is specified by a call to
    :func:`!field`, then the class attribute for this field will be

Doc/library/http.server.rst

@@ -458,55 +458,6 @@ such as using different index file names by overriding the class attribute
 :attr:`index_pages`.
 
 
-.. class:: CGIHTTPRequestHandler(request, client_address, server)
-
-   This class is used to serve either files or output of CGI scripts from the
-   current directory and below. Note that mapping HTTP hierarchic structure to
-   local directory structure is exactly as in :class:`SimpleHTTPRequestHandler`.
-
-   .. note::
-
-      CGI scripts run by the :class:`CGIHTTPRequestHandler` class cannot execute
-      redirects (HTTP code 302), because code 200 (script output follows) is
-      sent prior to execution of the CGI script.  This pre-empts the status
-      code.
-
-   The class will however, run the CGI script, instead of serving it as a file,
-   if it guesses it to be a CGI script.  Only directory-based CGI are used ---
-   the other common server configuration is to treat special extensions as
-   denoting CGI scripts.
-
-   The :func:`do_GET` and :func:`do_HEAD` functions are modified to run CGI scripts
-   and serve the output, instead of serving files, if the request leads to
-   somewhere below the ``cgi_directories`` path.
-
-   The :class:`CGIHTTPRequestHandler` defines the following data member:
-
-   .. attribute:: cgi_directories
-
-      This defaults to ``['/cgi-bin', '/htbin']`` and describes directories to
-      treat as containing CGI scripts.
-
-   The :class:`CGIHTTPRequestHandler` defines the following method:
-
-   .. method:: do_POST()
-
-      This method serves the ``'POST'`` request type, only allowed for CGI
-      scripts.  Error 501, "Can only POST to CGI scripts", is output when trying
-      to POST to a non-CGI url.
-
-   Note that CGI scripts will be run with UID of user nobody, for security
-   reasons.  Problems with the CGI script will be translated to error 403.
-
-   .. deprecated-removed:: 3.13 3.15
-
-      :class:`CGIHTTPRequestHandler` is being removed in 3.15.  CGI has not
-      been considered a good way to do things for well over a decade. This code
-      has been unmaintained for a while now and sees very little practical use.
-      Retaining it could lead to further :ref:`security considerations
-      <http.server-security>`.
-
-
 .. _http-server-cli:
 
 Command-line interface
@@ -563,24 +514,6 @@ The following options are accepted:
 
    .. versionadded:: 3.11
 
-.. option:: --cgi
-
-   :class:`CGIHTTPRequestHandler` can be enabled in the command line by passing
-   the ``--cgi`` option::
-
-      python -m http.server --cgi
-
-   .. deprecated-removed:: 3.13 3.15
-
-      :mod:`http.server` command line ``--cgi`` support is being removed
-      because :class:`CGIHTTPRequestHandler` is being removed.
-
-.. warning::
-
-   :class:`CGIHTTPRequestHandler` and the ``--cgi`` command-line option
-   are not intended for use by untrusted clients and may be vulnerable
-   to exploitation. Always use within a secure environment.
-
 .. option:: --tls-cert
 
    Specifies a TLS certificate chain for HTTPS connections::

Doc/library/json.rst

@@ -18,12 +18,17 @@ is a lightweight data interchange format inspired by
 `JavaScript <https://en.wikipedia.org/wiki/JavaScript>`_ object literal syntax
 (although it is not a strict subset of JavaScript [#rfc-errata]_ ).
 
+.. note::
+   The term "object" in the context of JSON processing in Python can be
+   ambiguous. All values in Python are objects. In JSON, an object refers to
+   any data wrapped in curly braces, similar to a Python dictionary.
+
 .. warning::
    Be cautious when parsing JSON data from untrusted sources. A malicious
    JSON string may cause the decoder to consume considerable CPU and memory
    resources. Limiting the size of data to be parsed is recommended.
 
-:mod:`json` exposes an API familiar to users of the standard library
+This module exposes an API familiar to users of the standard library
 :mod:`marshal` and :mod:`pickle` modules.
 
 Encoding basic Python object hierarchies::
@@ -60,7 +65,7 @@ Pretty printing::
         "6": 7
     }
 
-Specializing JSON object encoding::
+Customizing JSON object encoding::
 
    >>> import json
    >>> def custom_json(obj):
@@ -83,7 +88,7 @@ Decoding JSON::
     >>> json.load(io)
     ['streaming API']
 
-Specializing JSON object decoding::
+Customizing JSON object decoding::
 
     >>> import json
     >>> def as_complex(dct):
@@ -279,7 +284,7 @@ Basic Usage
 
    :param object_hook:
       If set, a function that is called with the result of
-      any object literal decoded (a :class:`dict`).
+      any JSON object literal decoded (a :class:`dict`).
       The return value of this function will be used
       instead of the :class:`dict`.
       This feature can be used to implement custom decoders,
@@ -289,7 +294,7 @@ Basic Usage
 
    :param object_pairs_hook:
       If set, a function that is called with the result of
-      any object literal decoded with an ordered list of pairs.
+      any JSON object literal decoded with an ordered list of pairs.
       The return value of this function will be used
       instead of the :class:`dict`.
       This feature can be used to implement custom decoders.

Doc/library/unittest.rst

@@ -109,7 +109,7 @@ Here is a short script to test three string methods::
       unittest.main()
 
 
-A testcase is created by subclassing :class:`unittest.TestCase`.  The three
+A test case is created by subclassing :class:`unittest.TestCase`.  The three
 individual tests are defined with methods whose names start with the letters
 ``test``.  This naming convention informs the test runner about which methods
 represent tests.

Doc/tutorial/controlflow.rst

@@ -999,7 +999,8 @@ scope::
    43
 
 The above example uses a lambda expression to return a function.  Another use
-is to pass a small function as an argument::
+is to pass a small function as an argument.  For instance, :meth:`list.sort`
+takes a sorting key function *key* which can be a lambda function::
 
    >>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
    >>> pairs.sort(key=lambda pair: pair[1])

Doc/whatsnew/3.10.rst

@@ -551,11 +551,12 @@ Patterns and classes
 
 If you are using classes to structure your data, you can use as a pattern
 the class name followed by an argument list resembling a constructor. This
-pattern has the ability to capture class attributes into variables::
+pattern has the ability to capture instance attributes into variables::
 
     class Point:
-        x: int
-        y: int
+        def __init__(self, x, y):
+            self.x = x
+            self.y = y
 
     def location(point):
         match point:

Doc/whatsnew/3.13.rst

@@ -1871,7 +1871,7 @@ New Deprecations
 
 * :mod:`http.server`:
 
-  * Deprecate :class:`~http.server.CGIHTTPRequestHandler`,
+  * Deprecate :class:`!CGIHTTPRequestHandler`,
     to be removed in Python 3.15.
     Process-based CGI HTTP servers have been out of favor for a very long time.
     This code was outdated, unmaintained, and rarely used.

Doc/whatsnew/3.15.rst

@@ -121,6 +121,15 @@ Deprecated
 Removed
 =======
 
+http.server
+-----------
+
+* Removed the :class:`!CGIHTTPRequestHandler` class
+  and the ``--cgi`` flag from the :program:`python -m http.server`
+  command-line interface. They were deprecated in Python 3.13.
+  (Contributed by Bénédikt Tran in :gh:`133810`.)
+
+
 platform
 --------