Merge branch 'python:main' into gh-93376

Author: sebbASF

Doc/glossary.rst

@@ -800,6 +800,10 @@ Glossary
       thread removes *key* from *mapping* after the test, but before the lookup.
       This issue can be solved with locks or by using the EAFP approach.
 
+   lexical analyzer
+
+      Formal name for the *tokenizer*; see :term:`token`.
+
    list
       A built-in Python :term:`sequence`.  Despite its name it is more akin
       to an array in other languages than to a linked list since access to
@@ -1291,6 +1295,17 @@ Glossary
       See also :term:`binary file` for a file object able to read and write
       :term:`bytes-like objects <bytes-like object>`.
 
+   token
+
+      A small unit of source code, generated by the
+      :ref:`lexical analyzer <lexical>` (also called the *tokenizer*).
+      Names, numbers, strings, operators,
+      newlines and similar are represented by tokens.
+
+      The :mod:`tokenize` module exposes Python's lexical analyzer.
+      The :mod:`token` module contains information on the various types
+      of tokens.
+
    triple-quoted string
       A string which is bound by three instances of either a quotation mark
       (") or an apostrophe (').  While they don't provide any functionality

Doc/library/bdb.rst

@@ -118,7 +118,7 @@ The :mod:`bdb` module also defines two classes:
 
       Count of the number of times a :class:`Breakpoint` has been hit.
 
-.. class:: Bdb(skip=None)
+.. class:: Bdb(skip=None, backend='settrace')
 
    The :class:`Bdb` class acts as a generic Python debugger base class.
 
@@ -132,9 +132,22 @@ The :mod:`bdb` module also defines two classes:
    frame is considered to originate in a certain module is determined
    by the ``__name__`` in the frame globals.
 
+   The *backend* argument specifies the backend to use for :class:`Bdb`. It
+   can be either ``'settrace'`` or ``'monitoring'``. ``'settrace'`` uses
+   :func:`sys.settrace` which has the best backward compatibility. The
+   ``'monitoring'`` backend uses the new :mod:`sys.monitoring` that was
+   introduced in Python 3.12, which can be much more efficient because it
+   can disable unused events. We are trying to keep the exact interfaces
+   for both backends, but there are some differences. The debugger developers
+   are encouraged to use the ``'monitoring'`` backend to achieve better
+   performance.
+
    .. versionchanged:: 3.1
       Added the *skip* parameter.
 
+   .. versionchanged:: 3.14
+      Added the *backend* parameter.
+
    The following methods of :class:`Bdb` normally don't need to be overridden.
 
    .. method:: canonic(filename)
@@ -146,6 +159,20 @@ The :mod:`bdb` module also defines two classes:
       <os.path.abspath>`. A *filename* with angle brackets, such as ``"<stdin>"``
       generated in interactive mode, is returned unchanged.
 
+   .. method:: start_trace(self)
+
+      Start tracing. For ``'settrace'`` backend, this method is equivalent to
+      ``sys.settrace(self.trace_dispatch)``
+
+      .. versionadded:: 3.14
+
+   .. method:: stop_trace(self)
+
+      Stop tracing. For ``'settrace'`` backend, this method is equivalent to
+      ``sys.settrace(None)``
+
+      .. versionadded:: 3.14
+
    .. method:: reset()
 
       Set the :attr:`!botframe`, :attr:`!stopframe`, :attr:`!returnframe` and
@@ -364,6 +391,28 @@ The :mod:`bdb` module also defines two classes:
       Return all breakpoints that are set.
 
 
+   Derived classes and clients can call the following methods to disable and
+   restart events to achieve better performance. These methods only work
+   when using the ``'monitoring'`` backend.
+
+   .. method:: disable_current_event()
+
+      Disable the current event until the next time :func:`restart_events` is
+      called. This is helpful when the debugger is not interested in the current
+      line.
+
+      .. versionadded:: 3.14
+
+   .. method:: restart_events()
+
+      Restart all the disabled events. This function is automatically called in
+      ``dispatch_*`` methods after ``user_*`` methods are called. If the
+      ``dispatch_*`` methods are not overridden, the disabled events will be
+      restarted after each user interaction.
+
+      .. versionadded:: 3.14
+
+
    Derived classes and clients can call the following methods to get a data
    structure representing a stack trace.
 

Doc/library/dis.rst

@@ -535,7 +535,7 @@ details of bytecode instructions as :class:`Instruction` instances:
       :class:`dis.Positions` object holding the
       start and end locations that are covered by this instruction.
 
-   .. data::cache_info
+   .. data:: cache_info
 
       Information about the cache entries of this instruction, as
       triplets of the form ``(name, size, data)``, where the ``name``

Doc/library/math.rst

@@ -350,8 +350,8 @@ Floating point manipulation functions
 
    *abs_tol* is the absolute tolerance; it defaults to ``0.0`` and it must be
    nonnegative.  When comparing ``x`` to ``0.0``, ``isclose(x, 0)`` is computed
-   as ``abs(x) <= rel_tol  * abs(x)``, which is ``False`` for any ``x`` and
-   rel_tol less than ``1.0``.  So add an appropriate positive abs_tol argument
+   as ``abs(x) <= rel_tol  * abs(x)``, which is ``False`` for any nonzero ``x`` and
+   *rel_tol* less than ``1.0``.  So add an appropriate positive *abs_tol* argument
    to the call.
 
    The IEEE 754 special values of ``NaN``, ``inf``, and ``-inf`` will be

Doc/library/pdb.rst

@@ -203,13 +203,32 @@ slightly different way:
    Enter post-mortem debugging of the exception found in
    :data:`sys.last_exc`.
 
+.. function:: set_default_backend(backend)
+
+   There are two supported backends for pdb: ``'settrace'`` and ``'monitoring'``.
+   See :class:`bdb.Bdb` for details. The user can set the default backend to
+   use if none is specified when instantiating :class:`Pdb`. If no backend is
+   specified, the default is ``'settrace'``.
+
+   .. note::
+
+      :func:`breakpoint` and :func:`set_trace` will not be affected by this
+      function. They always use ``'monitoring'`` backend.
+
+   .. versionadded:: 3.14
+
+.. function:: get_default_backend()
+
+   Returns the default backend for pdb.
+
+   .. versionadded:: 3.14
 
 The ``run*`` functions and :func:`set_trace` are aliases for instantiating the
 :class:`Pdb` class and calling the method of the same name.  If you want to
 access further features, you have to do this yourself:
 
 .. class:: Pdb(completekey='tab', stdin=None, stdout=None, skip=None, \
-               nosigint=False, readrc=True, mode=None)
+               nosigint=False, readrc=True, mode=None, backend=None)
 
    :class:`Pdb` is the debugger class.
 
@@ -235,6 +254,10 @@ access further features, you have to do this yourself:
    or ``None`` (for backwards compatible behaviour, as before the *mode*
    argument was added).
 
+   The *backend* argument specifies the backend to use for the debugger. If ``None``
+   is passed, the default backend will be used. See :func:`set_default_backend`.
+   Otherwise the supported backends are ``'settrace'`` and ``'monitoring'``.
+
    Example call to enable tracing with *skip*::
 
       import pdb; pdb.Pdb(skip=['django.*']).set_trace()
@@ -254,6 +277,9 @@ access further features, you have to do this yourself:
    .. versionadded:: 3.14
       Added the *mode* argument.
 
+   .. versionadded:: 3.14
+      Added the *backend* argument.
+
    .. versionchanged:: 3.14
       Inline breakpoints like :func:`breakpoint` or :func:`pdb.set_trace` will
       always stop the program at calling frame, ignoring the *skip* pattern (if any).

Doc/library/sqlite3.rst

@@ -290,9 +290,6 @@ Module functions
        :const:`PARSE_DECLTYPES` and :const:`PARSE_COLNAMES`
        to enable this.
        Column names takes precedence over declared types if both flags are set.
-       Types cannot be detected for generated fields (for example ``max(data)``),
-       even when the *detect_types* parameter is set; :class:`str` will be
-       returned instead.
        By default (``0``), type detection is disabled.
 
    :param isolation_level:
@@ -436,21 +433,6 @@ Module constants
    old style (pre-Python 3.12) transaction control behaviour.
    See :ref:`sqlite3-transaction-control-isolation-level` for more information.
 
-.. data:: PARSE_COLNAMES
-
-   Pass this flag value to the *detect_types* parameter of
-   :func:`connect` to look up a converter function by
-   using the type name, parsed from the query column name,
-   as the converter dictionary key.
-   The type name must be wrapped in square brackets (``[]``).
-
-   .. code-block:: sql
-
-      SELECT p as "p [point]" FROM test;  ! will look up converter "point"
-
-   This flag may be combined with :const:`PARSE_DECLTYPES` using the ``|``
-   (bitwise or) operator.
-
 .. data:: PARSE_DECLTYPES
 
    Pass this flag value to the *detect_types* parameter of
@@ -472,6 +454,27 @@ Module constants
    This flag may be combined with :const:`PARSE_COLNAMES` using the ``|``
    (bitwise or) operator.
 
+   .. note::
+
+      Generated fields (for example ``MAX(p)``) are returned as :class:`str`.
+      Use :const:`!PARSE_COLNAMES` to enforce types for such queries.
+
+.. data:: PARSE_COLNAMES
+
+   Pass this flag value to the *detect_types* parameter of
+   :func:`connect` to look up a converter function by
+   using the type name, parsed from the query column name,
+   as the converter dictionary key.
+   The query column name must be wrapped in double quotes (``"``)
+   and the type name must be wrapped in square brackets (``[]``).
+
+   .. code-block:: sql
+
+      SELECT MAX(p) as "p [point]" FROM test;  ! will look up converter "point"
+
+   This flag may be combined with :const:`PARSE_DECLTYPES` using the ``|``
+   (bitwise or) operator.
+
 .. data:: SQLITE_OK
           SQLITE_DENY
           SQLITE_IGNORE

Doc/library/urllib.request.rst

@@ -1249,7 +1249,10 @@ It is also possible to achieve the same result without using the
 
    >>> import urllib.request
    >>> f = urllib.request.urlopen('http://www.python.org/')
-   >>> print(f.read(100).decode('utf-8'))
+   >>> try:
+   ...     print(f.read(100).decode('utf-8'))
+   ... finally:
+   ...     f.close()
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtm
 
@@ -1294,7 +1297,8 @@ Use of Basic HTTP Authentication::
    opener = urllib.request.build_opener(auth_handler)
    # ...and install it globally so it can be used with urlopen.
    urllib.request.install_opener(opener)
-   urllib.request.urlopen('http://www.example.com/login.html')
+   with urllib.request.urlopen('http://www.example.com/login.html') as f:
+       print(f.read().decode('utf-8'))
 
 :func:`build_opener` provides many handlers by default, including a
 :class:`ProxyHandler`.  By default, :class:`ProxyHandler` uses the environment
@@ -1312,7 +1316,8 @@ programmatically supplied proxy URLs, and adds proxy authorization support with
 
    opener = urllib.request.build_opener(proxy_handler, proxy_auth_handler)
    # This time, rather than install the OpenerDirector, we use it directly:
-   opener.open('http://www.example.com/login.html')
+   with opener.open('http://www.example.com/login.html') as f:
+      print(f.read().decode('utf-8'))
 
 Adding HTTP headers:
 
@@ -1323,15 +1328,18 @@ Use the *headers* argument to the :class:`Request` constructor, or::
    req.add_header('Referer', 'http://www.python.org/')
    # Customize the default User-Agent header value:
    req.add_header('User-Agent', 'urllib-example/0.1 (Contact: . . .)')
-   r = urllib.request.urlopen(req)
+   with urllib.request.urlopen(req) as f:
+       print(f.read().decode('utf-8'))
+
 
 :class:`OpenerDirector` automatically adds a :mailheader:`User-Agent` header to
 every :class:`Request`.  To change this::
 
    import urllib.request
    opener = urllib.request.build_opener()
    opener.addheaders = [('User-agent', 'Mozilla/5.0')]
-   opener.open('http://www.example.com/')
+   with opener.open('http://www.example.com/') as f:
+      print(f.read().decode('utf-8'))
 
 Also, remember that a few standard headers (:mailheader:`Content-Length`,
 :mailheader:`Content-Type` and :mailheader:`Host`)

Doc/reference/compound_stmts.rst

@@ -420,16 +420,16 @@ is executed.  If there is a saved exception it is re-raised at the end of the
 :keyword:`!finally` clause.  If the :keyword:`!finally` clause raises another
 exception, the saved exception is set as the context of the new exception.
 If the :keyword:`!finally` clause executes a :keyword:`return`, :keyword:`break`
-or :keyword:`continue` statement, the saved exception is discarded::
+or :keyword:`continue` statement, the saved exception is discarded. For example,
+this function returns 42.
 
-   >>> def f():
-   ...     try:
-   ...         1/0
-   ...     finally:
-   ...         return 42
-   ...
-   >>> f()
-   42
+.. code-block::
+
+   def f():
+       try:
+           1/0
+       finally:
+           return 42
 
 The exception information is not available to the program during execution of
 the :keyword:`!finally` clause.
@@ -446,21 +446,25 @@ statement, the :keyword:`!finally` clause is also executed 'on the way out.'
 The return value of a function is determined by the last :keyword:`return`
 statement executed.  Since the :keyword:`!finally` clause always executes, a
 :keyword:`!return` statement executed in the :keyword:`!finally` clause will
-always be the last one executed::
+always be the last one executed. The following function returns 'finally'.
 
-   >>> def foo():
-   ...     try:
-   ...         return 'try'
-   ...     finally:
-   ...         return 'finally'
-   ...
-   >>> foo()
-   'finally'
+.. code-block::
+
+   def foo():
+       try:
+           return 'try'
+       finally:
+           return 'finally'
 
 .. versionchanged:: 3.8
    Prior to Python 3.8, a :keyword:`continue` statement was illegal in the
    :keyword:`!finally` clause due to a problem with the implementation.
 
+.. versionchanged:: next
+   The compiler emits a :exc:`SyntaxWarning` when a :keyword:`return`,
+   :keyword:`break` or :keyword:`continue` appears in a :keyword:`!finally`
+   block (see :pep:`765`).
+
 
 .. _with:
 .. _as:

Doc/reference/lexical_analysis.rst

@@ -8,8 +8,9 @@ Lexical analysis
 .. index:: lexical analysis, parser, token
 
 A Python program is read by a *parser*.  Input to the parser is a stream of
-*tokens*, generated by the *lexical analyzer*.  This chapter describes how the
-lexical analyzer breaks a file into tokens.
+:term:`tokens <token>`, generated by the *lexical analyzer* (also known as
+the *tokenizer*).
+This chapter describes how the lexical analyzer breaks a file into tokens.
 
 Python reads program text as Unicode code points; the encoding of a source file
 can be given by an encoding declaration and defaults to UTF-8, see :pep:`3120`