Merge branch 'main' into json-private-unicode-writer

Author: nineteendo

Doc/deprecations/pending-removal-in-3.15.rst

@@ -85,6 +85,13 @@ Pending removal in Python 3.15
     has been deprecated since Python 3.13.
     Use the class-based syntax or the functional syntax instead.
 
+  * When using the functional syntax of :class:`~typing.TypedDict`\s, failing
+    to pass a value to the *fields* parameter (``TD = TypedDict("TD")``) or
+    passing ``None`` (``TD = TypedDict("TD", None)``) has been deprecated
+    since Python 3.13.
+    Use ``class TD(TypedDict): pass`` or ``TD = TypedDict("TD", {})``
+    to create a TypedDict with zero field.
+
   * The :func:`typing.no_type_check_decorator` decorator function
     has been deprecated since Python 3.13.
     After eight years in the :mod:`typing` module,

Doc/library/annotationlib.rst

@@ -127,34 +127,34 @@ Classes
 
       Values are the result of evaluating the annotation expressions.
 
-   .. attribute:: FORWARDREF
+   .. attribute:: VALUE_WITH_FAKE_GLOBALS
       :value: 2
 
+      Special value used to signal that an annotate function is being
+      evaluated in a special environment with fake globals. When passed this
+      value, annotate functions should either return the same value as for
+      the :attr:`Format.VALUE` format, or raise :exc:`NotImplementedError`
+      to signal that they do not support execution in this environment.
+      This format is only used internally and should not be passed to
+      the functions in this module.
+
+   .. attribute:: FORWARDREF
+      :value: 3
+
       Values are real annotation values (as per :attr:`Format.VALUE` format)
       for defined values, and :class:`ForwardRef` proxies for undefined
       values. Real objects may contain references to :class:`ForwardRef`
       proxy objects.
 
    .. attribute:: STRING
-      :value: 3
+      :value: 4
 
       Values are the text string of the annotation as it appears in the
       source code, up to modifications including, but not restricted to,
       whitespace normalizations and constant values optimizations.
 
       The exact values of these strings may change in future versions of Python.
 
-   .. attribute:: VALUE_WITH_FAKE_GLOBALS
-      :value: 4
-
-      Special value used to signal that an annotate function is being
-      evaluated in a special environment with fake globals. When passed this
-      value, annotate functions should either return the same value as for
-      the :attr:`Format.VALUE` format, or raise :exc:`NotImplementedError`
-      to signal that they do not support execution in this environment.
-      This format is only used internally and should not be passed to
-      the functions in this module.
-
    .. versionadded:: 3.14
 
 .. class:: ForwardRef
@@ -485,3 +485,117 @@ annotations from the class and puts them in a separate attribute:
          typ.classvars = classvars  # Store the ClassVars in a separate attribute
          return typ
 
+
+Limitations of the ``STRING`` format
+------------------------------------
+
+The :attr:`~Format.STRING` format is meant to approximate the source code
+of the annotation, but the implementation strategy used means that it is not
+always possible to recover the exact source code.
+
+First, the stringifier of course cannot recover any information that is not present in
+the compiled code, including comments, whitespace, parenthesization, and operations that
+get simplified by the compiler.
+
+Second, the stringifier can intercept almost all operations that involve names looked
+up in some scope, but it cannot intercept operations that operate fully on constants.
+As a corollary, this also means it is not safe to request the ``STRING`` format on
+untrusted code: Python is powerful enough that it is possible to achieve arbitrary
+code execution even with no access to any globals or builtins. For example:
+
+.. code-block:: pycon
+
+  >>> def f(x: (1).__class__.__base__.__subclasses__()[-1].__init__.__builtins__["print"]("Hello world")): pass
+  ...
+  >>> annotationlib.get_annotations(f, format=annotationlib.Format.SOURCE)
+  Hello world
+  {'x': 'None'}
+
+.. note::
+   This particular example works as of the time of writing, but it relies on
+   implementation details and is not guaranteed to work in the future.
+
+Among the different kinds of expressions that exist in Python,
+as represented by the :mod:`ast` module, some expressions are supported,
+meaning that the ``STRING`` format can generally recover the original source code;
+others are unsupported, meaning that they may result in incorrect output or an error.
+
+The following are supported (sometimes with caveats):
+
+* :class:`ast.BinOp`
+* :class:`ast.UnaryOp`
+
+  * :class:`ast.Invert` (``~``), :class:`ast.UAdd` (``+``), and :class:`ast.USub` (``-``) are supported
+  * :class:`ast.Not` (``not``) is not supported
+
+* :class:`ast.Dict` (except when using ``**`` unpacking)
+* :class:`ast.Set`
+* :class:`ast.Compare`
+
+  * :class:`ast.Eq` and :class:`ast.NotEq` are supported
+  * :class:`ast.Lt`, :class:`ast.LtE`, :class:`ast.Gt`, and :class:`ast.GtE` are supported, but the operand may be flipped
+  * :class:`ast.Is`, :class:`ast.IsNot`, :class:`ast.In`, and :class:`ast.NotIn` are not supported
+
+* :class:`ast.Call` (except when using ``**`` unpacking)
+* :class:`ast.Constant` (though not the exact representation of the constant; for example, escape
+  sequences in strings are lost; hexadecimal numbers are converted to decimal)
+* :class:`ast.Attribute` (assuming the value is not a constant)
+* :class:`ast.Subscript` (assuming the value is not a constant)
+* :class:`ast.Starred` (``*`` unpacking)
+* :class:`ast.Name`
+* :class:`ast.List`
+* :class:`ast.Tuple`
+* :class:`ast.Slice`
+
+The following are unsupported, but throw an informative error when encountered by the
+stringifier:
+
+* :class:`ast.FormattedValue` (f-strings; error is not detected if conversion specifiers like ``!r``
+  are used)
+* :class:`ast.JoinedStr` (f-strings)
+
+The following are unsupported and result in incorrect output:
+
+* :class:`ast.BoolOp` (``and`` and ``or``)
+* :class:`ast.IfExp`
+* :class:`ast.Lambda`
+* :class:`ast.ListComp`
+* :class:`ast.SetComp`
+* :class:`ast.DictComp`
+* :class:`ast.GeneratorExp`
+
+The following are disallowed in annotation scopes and therefore not relevant:
+
+* :class:`ast.NamedExpr` (``:=``)
+* :class:`ast.Await`
+* :class:`ast.Yield`
+* :class:`ast.YieldFrom`
+
+
+Limitations of the ``FORWARDREF`` format
+----------------------------------------
+
+The :attr:`~Format.FORWARDREF` format aims to produce real values as much
+as possible, with anything that cannot be resolved replaced with
+:class:`ForwardRef` objects. It is affected by broadly the same Limitations
+as the :attr:`~Format.STRING` format: annotations that perform operations on
+literals or that use unsupported expression types may raise exceptions when
+evaluated using the :attr:`~Format.FORWARDREF` format.
+
+Below are a few examples of the behavior with unsupported expressions:
+
+.. code-block:: pycon
+
+   >>> from annotationlib import get_annotations, Format
+   >>> def zerodiv(x: 1 / 0): ...
+   >>> get_annotations(zerodiv, format=Format.STRING)
+   Traceback (most recent call last):
+     ...
+   ZeroDivisionError: division by zero
+   >>> get_annotations(zerodiv, format=Format.FORWARDREF)
+   Traceback (most recent call last):
+     ...
+   ZeroDivisionError: division by zero
+   >>> def ifexp(x: 1 if y else 0): ...
+   >>> get_annotations(ifexp, format=Format.STRING)
+   {'x': '1'}

Doc/library/datetime.rst

@@ -261,6 +261,22 @@ A :class:`timedelta` object represents a duration, the difference between two
       >>> (d.days, d.seconds, d.microseconds)
       (-1, 86399, 999999)
 
+   Since the string representation of :class:`!timedelta` objects can be confusing,
+   use the following recipe to produce a more readable format:
+
+   .. code-block:: pycon
+
+      >>> def pretty_timedelta(td):
+      ...     if td.days >= 0:
+      ...         return str(td)
+      ...     return f'-({-td!s})'
+      ...
+      >>> d = timedelta(hours=-1)
+      >>> str(d)  # not human-friendly
+      '-1 day, 23:00:00'
+      >>> pretty_timedelta(d)
+      '-(1:00:00)'
+
 
 Class attributes:
 

Doc/library/heapq.rst

@@ -315,6 +315,7 @@ The strange invariant above is meant to be an efficient memory representation
 for a tournament.  The numbers below are *k*, not ``a[k]``:
 
 .. figure:: heapq-binary-tree.svg
+   :class: invert-in-dark-mode
    :align: center
    :alt: Example (min-heap) binary tree.
 

Doc/library/math.rst

@@ -144,8 +144,7 @@ Number-theoretic functions
 
 .. function:: factorial(n)
 
-   Return *n* factorial as an integer.  Raises :exc:`ValueError` if *n* is not integral or
-   is negative.
+   Return factorial of the nonnegative integer *n*.
 
    .. versionchanged:: 3.10
       Floats with integral values (like ``5.0``) are no longer accepted.

Doc/library/unittest.mock.rst

@@ -2654,9 +2654,9 @@ with any methods on the mock:
 
 .. code-block:: pycon
 
-    >>> mock.has_data()
+    >>> mock.header_items()
     <mock.Mock object at 0x...>
-    >>> mock.has_data.assret_called_with()  # Intentional typo!
+    >>> mock.header_items.assret_called_with()  # Intentional typo!
 
 Auto-speccing solves this problem. You can either pass ``autospec=True`` to
 :func:`patch` / :func:`patch.object` or use the :func:`create_autospec` function to create a

Doc/library/urllib.request.rst

@@ -1121,7 +1121,7 @@ HTTPHandler Objects
 .. method:: HTTPHandler.http_open(req)
 
    Send an HTTP request, which can be either GET or POST, depending on
-   ``req.has_data()``.
+   ``req.data``.
 
 
 .. _https-handler-objects:
@@ -1133,7 +1133,7 @@ HTTPSHandler Objects
 .. method:: HTTPSHandler.https_open(req)
 
    Send an HTTPS request, which can be either GET or POST, depending on
-   ``req.has_data()``.
+   ``req.data``.
 
 
 .. _file-handler-objects:

Doc/reference/compound_stmts.rst

@@ -1885,7 +1885,7 @@ expressions. The presence of annotations does not change the runtime semantics o
 the code, except if some mechanism is used that introspects and uses the annotations
 (such as :mod:`dataclasses` or :func:`functools.singledispatch`).
 
-By default, annotations are lazily evaluated in a :ref:`annotation scope <annotation-scopes>`.
+By default, annotations are lazily evaluated in an :ref:`annotation scope <annotation-scopes>`.
 This means that they are not evaluated when the code containing the annotation is evaluated.
 Instead, the interpreter saves information that can be used to evaluate the annotation later
 if requested. The :mod:`annotationlib` module provides tools for evaluating annotations.
@@ -1898,6 +1898,12 @@ all annotations are instead stored as strings::
    >>> f.__annotations__
    {'param': 'annotation'}
 
+This future statement will be deprecated and removed in a future version of Python,
+but not before Python 3.13 reaches its end of life (see :pep:`749`).
+When it is used, introspection tools like
+:func:`annotationlib.get_annotations` and :func:`typing.get_type_hints` are
+less likely to be able to resolve annotations at runtime.
+
 
 .. rubric:: Footnotes
 

Doc/using/windows.rst

@@ -77,15 +77,10 @@ To install the file downloaded from python.org, either double-click and select
 "Install", or run ``Add-AppxPackage <path to MSIX>`` in Windows Powershell.
 
 After installation, the ``python``, ``py``, and ``pymanager`` commands should be
-available. If they are not, click Start and search for "Manage app execution
-aliases". This settings page will let you enable the relevant commands. They
-will be labelled "Python (default)", "Python (default windowed)", and "Python
-install manager".
-
-If you have existing installations of Python, or you have modified your
-:envvar:`PATH` variable, you may need to remove them or undo the modifications
-in order for the commands to work. Old versions of Python can be reinstalled
-using the Python install manager.
+available. If you have existing installations of Python, or you have modified
+your :envvar:`PATH` variable, you may need to remove them or undo the
+modifications. See :ref:`pymanager-troubleshoot` for more help with fixing
+non-working commands.
 
 When you first install a runtime, you will likely be prompted to add a directory
 to your :envvar:`PATH`. This is optional, if you prefer to use the ``py``
@@ -150,6 +145,10 @@ want to be passed to the runtime (such as script files or the module to launch):
    $> py -m this
    ...
 
+The default runtime can be overridden with the :envvar:`PYTHON_MANAGER_DEFAULT`
+environment variable, or a configuration file. See :ref:`pymanager-config` for
+information about configuration settings.
+
 To launch a specific runtime, the ``py`` command accepts a ``-V:<TAG>`` option.
 This option must be specified before any others. The tag is part or all of the
 identifier for the runtime; for those from the CPython team, it looks like the
@@ -472,6 +471,10 @@ directory (which you may have added to your :envvar:`PATH` environment variable)
 can be used in a shebang, even if it is not on your :envvar:`PATH`. This allows
 the use of shebangs like ``/usr/bin/python3.12`` to select a particular runtime.
 
+If no runtimes are installed, or if automatic installation is enabled, the
+requested runtime will be installed if necessary. See :ref:`pymanager-config`
+for information about configuration settings.
+
 The ``/usr/bin/env`` form of shebang line will also search the :envvar:`PATH`
 environment variable for unrecognized commands. This corresponds to the
 behaviour of the Unix ``env`` program, which performs the same search, but

Doc/whatsnew/3.14.rst

@@ -82,7 +82,7 @@ and improvements in user-friendliness and correctness.
 
 .. PEP-sized items next.
 
-* :ref:`PEP 649: deferred evaluation of annotations <whatsnew314-pep649>`
+* :ref:`PEP 649 and 749: deferred evaluation of annotations <whatsnew314-pep649>`
 * :ref:`PEP 741: Python Configuration C API <whatsnew314-pep741>`
 * :ref:`PEP 750: Template strings <whatsnew314-pep750>`
 * :ref:`PEP 758: Allow except and except* expressions without parentheses <whatsnew314-pep758>`
@@ -362,18 +362,19 @@ Check :pep:`758` for more details.
 
 .. _whatsnew314-pep649:
 
-PEP 649: deferred evaluation of annotations
--------------------------------------------
+PEP 649 and 749: deferred evaluation of annotations
+---------------------------------------------------
 
 The :term:`annotations <annotation>` on functions, classes, and modules are no
 longer evaluated eagerly. Instead, annotations are stored in special-purpose
 :term:`annotate functions <annotate function>` and evaluated only when
-necessary. This is specified in :pep:`649` and :pep:`749`.
+necessary (except if ``from __future__ import annotations`` is used).
+This is specified in :pep:`649` and :pep:`749`.
 
 This change is designed to make annotations in Python more performant and more
 usable in most circumstances. The runtime cost for defining annotations is
 minimized, but it remains possible to introspect annotations at runtime.
-It is usually no longer necessary to enclose annotations in strings if they
+It is no longer necessary to enclose annotations in strings if they
 contain forward references.
 
 The new :mod:`annotationlib` module provides tools for inspecting deferred
@@ -409,7 +410,8 @@ writing annotations the same way you did with previous versions of Python.
 You will likely be able to remove quoted strings in annotations, which are frequently
 used for forward references. Similarly, if you use ``from __future__ import annotations``
 to avoid having to write strings in annotations, you may well be able to
-remove that import. However, if you rely on third-party libraries that read annotations,
+remove that import once you support only Python 3.14 and newer.
+However, if you rely on third-party libraries that read annotations,
 those libraries may need changes to support unquoted annotations before they
 work as expected.
 
@@ -422,6 +424,11 @@ annotations. For example, you may want to use :func:`annotationlib.get_annotatio
 with the :attr:`~annotationlib.Format.FORWARDREF` format, as the :mod:`dataclasses`
 module now does.
 
+The external :pypi:`typing_extensions` package provides partial backports of some of the
+functionality of the :mod:`annotationlib` module, such as the :class:`~annotationlib.Format`
+enum and the :func:`~annotationlib.get_annotations` function. These can be used to
+write cross-version code that takes advantage of the new behavior in Python 3.14.
+
 Related changes
 ^^^^^^^^^^^^^^^
 
@@ -433,6 +440,10 @@ functions in the standard library, there are many ways in which your code may
 not work in Python 3.14. To safeguard your code against future changes,
 use only the documented functionality of the :mod:`annotationlib` module.
 
+In particular, do not read annotations directly from the namespace dictionary
+attribute of type objects. Use :func:`annotationlib.get_annotate_from_class_namespace`
+during class construction and :func:`annotationlib.get_annotations` afterwards.
+
 ``from __future__ import annotations``
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -444,8 +455,10 @@ Python without deferred evaluation of annotations, reaches its end of life in 20
 In Python 3.14, the behavior of code using ``from __future__ import annotations``
 is unchanged.
 
+(Contributed by Jelle Zijlstra in :gh:`119180`; :pep:`649` was written by Larry Hastings.)
+
 .. seealso::
-   :pep:`649`.
+   :pep:`649` and :pep:`749`.
 
 
 Improved error messages