Merge branch '3.12' into bp-312/17d06aeb5476099bc1acd89cd6f69e239e0f9350/enum-131045

Author: ethanfurman

Doc/c-api/allocation.rst

@@ -35,6 +35,10 @@ Allocating Objects on the Heap
    The size of the memory allocation is determined from the
    :c:member:`~PyTypeObject.tp_basicsize` field of the type object.
 
+   Note that this function is unsuitable if *typeobj* has
+   :c:macro:`Py_TPFLAGS_HAVE_GC` set. For such objects,
+   use :c:func:`PyObject_GC_New` instead.
+
 
 .. c:macro:: PyObject_NewVar(TYPE, typeobj, size)
 
@@ -49,6 +53,10 @@ Allocating Objects on the Heap
    fields into the same allocation decreases the number of allocations,
    improving the memory management efficiency.
 
+   Note that this function is unsuitable if *typeobj* has
+   :c:macro:`Py_TPFLAGS_HAVE_GC` set. For such objects,
+   use :c:func:`PyObject_GC_NewVar` instead.
+
 
 .. c:function:: void PyObject_Del(void *op)
 

Doc/c-api/long.rst

@@ -135,6 +135,16 @@ distinguished from a number.  Use :c:func:`PyErr_Occurred` to disambiguate.
    .. versionchanged:: 3.10
       This function will no longer use :meth:`~object.__int__`.
 
+   .. c:namespace:: NULL
+
+   .. c:function:: long PyLong_AS_LONG(PyObject *obj)
+
+      A :term:`soft deprecated` alias.
+      Exactly equivalent to the preferred ``PyLong_AsLong``. In particular,
+      it can fail with :exc:`OverflowError` or another exception.
+
+      .. deprecated:: 3.14
+         The function is soft deprecated.
 
 .. c:function:: long PyLong_AsLongAndOverflow(PyObject *obj, int *overflow)
 

Doc/c-api/type.rst

@@ -397,6 +397,9 @@ The following functions and structs are used to create
       class need *in addition* to the superclass.
       Use :c:func:`PyObject_GetTypeData` to get a pointer to subclass-specific
       memory reserved this way.
+      For negative :c:member:`!basicsize`, Python will insert padding when
+      needed to meet :c:member:`~PyTypeObject.tp_basicsize`'s alignment
+      requirements.
 
       .. versionchanged:: 3.12
 

Doc/c-api/typeobj.rst

@@ -473,7 +473,7 @@ PyTypeObject Definition
 -----------------------
 
 The structure definition for :c:type:`PyTypeObject` can be found in
-:file:`Include/object.h`.  For convenience of reference, this repeats the
+:file:`Include/cpython/object.h`.  For convenience of reference, this repeats the
 definition found there:
 
 .. XXX Drop this?
@@ -559,6 +559,9 @@ PyVarObject Slots
    initialized to zero. For :ref:`dynamically allocated type objects
    <heap-types>`, this field has a special internal meaning.
 
+   This field should be accessed using the :c:func:`Py_SIZE()` and
+   :c:func:`Py_SET_SIZE()` macros.
+
    **Inheritance:**
 
    This field is not inherited by subtypes.
@@ -609,47 +612,86 @@ and :c:data:`PyType_Type` effectively act as defaults.)
 
 
 .. c:member:: Py_ssize_t PyTypeObject.tp_basicsize
-             Py_ssize_t PyTypeObject.tp_itemsize
+              Py_ssize_t PyTypeObject.tp_itemsize
 
    These fields allow calculating the size in bytes of instances of the type.
 
    There are two kinds of types: types with fixed-length instances have a zero
-   :c:member:`~PyTypeObject.tp_itemsize` field, types with variable-length instances have a non-zero
-   :c:member:`~PyTypeObject.tp_itemsize` field.  For a type with fixed-length instances, all
-   instances have the same size, given in :c:member:`~PyTypeObject.tp_basicsize`.
+   :c:member:`!tp_itemsize` field, types with variable-length instances have a non-zero
+   :c:member:`!tp_itemsize` field.  For a type with fixed-length instances, all
+   instances have the same size, given in :c:member:`!tp_basicsize`.
+   (Exceptions to this rule can be made using
+   :c:func:`PyUnstable_Object_GC_NewWithExtraData`.)
 
    For a type with variable-length instances, the instances must have an
-   :c:member:`~PyVarObject.ob_size` field, and the instance size is :c:member:`~PyTypeObject.tp_basicsize` plus N
-   times :c:member:`~PyTypeObject.tp_itemsize`, where N is the "length" of the object.  The value of
-   N is typically stored in the instance's :c:member:`~PyVarObject.ob_size` field.  There are
-   exceptions:  for example, ints use a negative :c:member:`~PyVarObject.ob_size` to indicate a
-   negative number, and N is ``abs(ob_size)`` there.  Also, the presence of an
-   :c:member:`~PyVarObject.ob_size` field in the instance layout doesn't mean that the instance
-   structure is variable-length (for example, the structure for the list type has
-   fixed-length instances, yet those instances have a meaningful :c:member:`~PyVarObject.ob_size`
-   field).
-
-   The basic size includes the fields in the instance declared by the macro
-   :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD` (whichever is used to
-   declare the instance struct) and this in turn includes the  :c:member:`~PyObject._ob_prev` and
-   :c:member:`~PyObject._ob_next` fields if they are present.  This means that the only correct
-   way to get an initializer for the :c:member:`~PyTypeObject.tp_basicsize` is to use the
-   ``sizeof`` operator on the struct used to declare the instance layout.
-   The basic size does not include the GC header size.
+   :c:member:`~PyVarObject.ob_size` field, and the instance size is
+   :c:member:`!tp_basicsize` plus N times :c:member:`!tp_itemsize`,
+   where N is the "length" of the object.
+
+   Functions like :c:func:`PyObject_NewVar` will take the value of N as an
+   argument, and store in the instance's :c:member:`~PyVarObject.ob_size` field.
+   Note that the :c:member:`~PyVarObject.ob_size` field may later be used for
+   other purposes. For example, :py:type:`int` instances use the bits of
+   :c:member:`~PyVarObject.ob_size` in an implementation-defined
+   way; the underlying storage and its size should be acessed using
+   :c:func:`PyLong_Export`.
+
+   .. note::
 
-   A note about alignment: if the variable items require a particular alignment,
-   this should be taken care of by the value of :c:member:`~PyTypeObject.tp_basicsize`.  Example:
-   suppose a type implements an array of ``double``. :c:member:`~PyTypeObject.tp_itemsize` is
-   ``sizeof(double)``. It is the programmer's responsibility that
-   :c:member:`~PyTypeObject.tp_basicsize` is a multiple of ``sizeof(double)`` (assuming this is the
-   alignment requirement for ``double``).
+      The :c:member:`~PyVarObject.ob_size` field should be accessed using
+      the :c:func:`Py_SIZE()` and :c:func:`Py_SET_SIZE()` macros.
 
-   For any type with variable-length instances, this field must not be ``NULL``.
+   Also, the presence of an :c:member:`~PyVarObject.ob_size` field in the
+   instance layout doesn't mean that the instance structure is variable-length.
+   For example, the :py:type:`list` type has fixed-length instances, yet those
+   instances have a :c:member:`~PyVarObject.ob_size` field.
+   (As with :py:type:`int`, avoid reading lists' :c:member:`!ob_size` directly.
+   Call :c:func:`PyList_Size` instead.)
+
+   The :c:member:`!tp_basicsize` includes size needed for data of the type's
+   :c:member:`~PyTypeObject.tp_base`, plus any extra data needed
+   by each instance.
+
+   The  correct way to set :c:member:`!tp_basicsize` is to use the
+   ``sizeof`` operator on the struct used to declare the instance layout.
+   This struct must include the struct used to declare the base type.
+   In other words, :c:member:`!tp_basicsize` must be greater than or equal
+   to the base's :c:member:`!tp_basicsize`.
+
+   Since every type is a subtype of :py:type:`object`, this struct must
+   include :c:type:`PyObject` or :c:type:`PyVarObject` (depending on
+   whether :c:member:`~PyVarObject.ob_size` should be included). These are
+   usually defined by the macro :c:macro:`PyObject_HEAD` or
+   :c:macro:`PyObject_VAR_HEAD`, respectively.
+
+   The basic size does not include the GC header size, as that header is not
+   part of :c:macro:`PyObject_HEAD`.
+
+   For cases where struct used to declare the base type is unknown,
+   see :c:member:`PyType_Spec.basicsize` and :c:func:`PyType_FromMetaclass`.
+
+   Notes about alignment:
+
+   - :c:member:`!tp_basicsize` must be a multiple of ``_Alignof(PyObject)``.
+     When using ``sizeof`` on a ``struct`` that includes
+     :c:macro:`PyObject_HEAD`, as recommended, the compiler ensures this.
+     When not using a C ``struct``, or when using compiler
+     extensions like ``__attribute__((packed))``, it is up to you.
+   - If the variable items require a particular alignment,
+     :c:member:`!tp_basicsize` and :c:member:`!tp_itemsize` must each be a
+     multiple of that alignment.
+     For example, if a type's variable part stores a ``double``, it is
+     your responsibility that both fields are a multiple of
+     ``_Alignof(double)``.
 
    **Inheritance:**
 
-   These fields are inherited separately by subtypes.  If the base type has a
-   non-zero :c:member:`~PyTypeObject.tp_itemsize`, it is generally not safe to set
+   These fields are inherited separately by subtypes.
+   (That is, if the field is set to zero, :c:func:`PyType_Ready` will copy
+   the value from the base type, indicating that the instances do not
+   need additional storage.)
+
+   If the base type has a non-zero :c:member:`~PyTypeObject.tp_itemsize`, it is generally not safe to set
    :c:member:`~PyTypeObject.tp_itemsize` to a different non-zero value in a subtype (though this
    depends on the implementation of the base type).
 

Doc/glossary.rst

@@ -734,6 +734,10 @@ Glossary
       thread removes *key* from *mapping* after the test, but before the lookup.
       This issue can be solved with locks or by using the EAFP approach.
 
+   lexical analyzer
+
+      Formal name for the *tokenizer*; see :term:`token`.
+
    list
       A built-in Python :term:`sequence`.  Despite its name it is more akin
       to an array in other languages than to a linked list since access to
@@ -1212,6 +1216,17 @@ Glossary
       See also :term:`binary file` for a file object able to read and write
       :term:`bytes-like objects <bytes-like object>`.
 
+   token
+
+      A small unit of source code, generated by the
+      :ref:`lexical analyzer <lexical>` (also called the *tokenizer*).
+      Names, numbers, strings, operators,
+      newlines and similar are represented by tokens.
+
+      The :mod:`tokenize` module exposes Python's lexical analyzer.
+      The :mod:`token` module contains information on the various types
+      of tokens.
+
    triple-quoted string
       A string which is bound by three instances of either a quotation mark
       (") or an apostrophe (').  While they don't provide any functionality

Doc/library/cmdline.rst

@@ -13,7 +13,7 @@ The following modules have a command-line interface.
 * :mod:`cProfile`: see :ref:`profile <profile-cli>`
 * :ref:`difflib <difflib-interface>`
 * :ref:`dis <dis-cli>`
-* :mod:`doctest`
+* :ref:`doctest <doctest-cli>`
 * :mod:`!encodings.rot_13`
 * :mod:`ensurepip`
 * :mod:`filecmp`

Doc/library/copy.rst

@@ -80,13 +80,22 @@ pickle functions from the :mod:`copyreg` module.
    single: __deepcopy__() (copy protocol)
 
 In order for a class to define its own copy implementation, it can define
-special methods :meth:`__copy__` and :meth:`__deepcopy__`.  The former is called
-to implement the shallow copy operation; no additional arguments are passed.
-The latter is called to implement the deep copy operation; it is passed one
-argument, the ``memo`` dictionary.  If the :meth:`__deepcopy__` implementation needs
-to make a deep copy of a component, it should call the :func:`deepcopy` function
-with the component as first argument and the memo dictionary as second argument.
-The memo dictionary should be treated as an opaque object.
+special methods :meth:`~object.__copy__` and :meth:`~object.__deepcopy__`.
+
+.. method:: object.__copy__(self)
+   :noindexentry:
+
+   Called to implement the shallow copy operation;
+   no additional arguments are passed.
+
+.. method:: object.__deepcopy__(self, memo)
+   :noindexentry:
+
+   Called to implement the deep copy operation; it is passed one
+   argument, the *memo* dictionary.  If the ``__deepcopy__`` implementation needs
+   to make a deep copy of a component, it should call the :func:`deepcopy` function
+   with the component as first argument and the *memo* dictionary as second argument.
+   The *memo* dictionary should be treated as an opaque object.
 
 
 .. seealso::

Doc/library/datetime.rst

@@ -2526,7 +2526,24 @@ Broadly speaking, ``d.strftime(fmt)`` acts like the :mod:`time` module's
 
 For the :meth:`.datetime.strptime` class method, the default value is
 ``1900-01-01T00:00:00.000``: any components not specified in the format string
-will be pulled from the default value. [#]_
+will be pulled from the default value.
+
+.. note::
+   When used to parse partial dates lacking a year, :meth:`~.datetime.strptime`
+   will raise when encountering February 29 because its default year of 1900 is
+   *not* a leap year.  Always add a default leap year to partial date strings
+   before parsing.
+
+.. doctest::
+
+    >>> from datetime import datetime
+    >>> value = "2/29"
+    >>> datetime.strptime(value, "%m/%d")
+    Traceback (most recent call last):
+    ...
+    ValueError: day is out of range for month
+    >>> datetime.strptime(f"1904 {value}", "%Y %m/%d")
+    datetime.datetime(1904, 2, 29, 0, 0)
 
 Using ``datetime.strptime(date_string, format)`` is equivalent to::
 
@@ -2652,6 +2669,11 @@ Notes:
    for  formats ``%d``, ``%m``, ``%H``, ``%I``, ``%M``, ``%S``, ``%j``, ``%U``,
    ``%W``, and ``%V``. Format ``%y`` does require a leading zero.
 
+(10)
+   Parsing dates without a year using :meth:`~.datetime.strptime` will fail on
+   representations of February 29 as that date does not exist in the default
+   year of 1900.
+
 .. rubric:: Footnotes
 
 .. [#] If, that is, we ignore the effects of Relativity
@@ -2665,5 +2687,3 @@ Notes:
 .. [#] See R. H. van Gent's `guide to the mathematics of the ISO 8601 calendar
        <https://web.archive.org/web/20220531051136/https://webspace.science.uu.nl/~gent0113/calendar/isocalendar.htm>`_
        for a good explanation.
-
-.. [#] Passing ``datetime.strptime('Feb 29', '%b %d')`` will fail since 1900 is not a leap year.

Doc/library/doctest.rst

@@ -173,15 +173,8 @@ prohibit it by passing ``verbose=False``.  In either of those cases,
 ``sys.argv`` is not examined by :func:`testmod` (so passing ``-v`` or not
 has no effect).
 
-There is also a command line shortcut for running :func:`testmod`.  You can
-instruct the Python interpreter to run the doctest module directly from the
-standard library and pass the module name(s) on the command line::
-
-   python -m doctest -v example.py
-
-This will import :file:`example.py` as a standalone module and run
-:func:`testmod` on it.  Note that this may not work correctly if the file is
-part of a package and imports other submodules from that package.
+There is also a command line shortcut for running :func:`testmod`, see section
+:ref:`doctest-cli`.
 
 For more information on :func:`testmod`, see section :ref:`doctest-basic-api`.
 
@@ -244,16 +237,53 @@ Like :func:`testmod`, :func:`testfile`'s verbosity can be set with the
 ``-v`` command-line switch or with the optional keyword argument
 *verbose*.
 
-There is also a command line shortcut for running :func:`testfile`.  You can
-instruct the Python interpreter to run the doctest module directly from the
-standard library and pass the file name(s) on the command line::
+There is also a command line shortcut for running :func:`testfile`, see section
+:ref:`doctest-cli`.
 
-   python -m doctest -v example.txt
+For more information on :func:`testfile`, see section :ref:`doctest-basic-api`.
 
-Because the file name does not end with :file:`.py`, :mod:`doctest` infers that
-it must be run with :func:`testfile`, not :func:`testmod`.
 
-For more information on :func:`testfile`, see section :ref:`doctest-basic-api`.
+.. _doctest-cli:
+
+Command-line Usage
+------------------
+
+The :mod:`doctest` module can be invoked as a script from the command line:
+
+.. code-block:: bash
+
+   python -m doctest [-v] [-o OPTION] [-f] file [file ...]
+
+.. program:: doctest
+
+.. option:: -v, --verbose
+
+   Detailed report of all examples tried is printed to standard output,
+   along with assorted summaries at the end::
+
+      python -m doctest -v example.py
+
+   This will import :file:`example.py` as a standalone module and run
+   :func:`testmod` on it. Note that this may not work correctly if the
+   file is part of a package and imports other submodules from that package.
+
+   If the file name does not end with :file:`.py`, :mod:`!doctest` infers
+   that it must be run with :func:`testfile` instead::
+
+      python -m doctest -v example.txt
+
+.. option:: -o, --option <option>
+
+   Option flags control various aspects of doctest's behavior, see section
+   :ref:`doctest-options`.
+
+   .. versionadded:: 3.4
+
+.. option:: -f, --fail-fast
+
+   This is shorthand for ``-o FAIL_FAST``.
+
+   .. versionadded:: 3.4
 
 
 .. _doctest-how-it-works:
@@ -536,9 +566,6 @@ Symbolic names for the flags are supplied as module constants, which can be
 The names can also be used in :ref:`doctest directives <doctest-directives>`,
 and may be passed to the doctest command line interface via the ``-o`` option.
 
-.. versionadded:: 3.4
-   The ``-o`` command line option.
-
 The first group of options define test semantics, controlling aspects of how
 doctest decides whether actual output matches an example's expected output:
 
@@ -678,11 +705,6 @@ The second group of options controls how test failures are reported:
    1.  This flag may be useful during debugging, since examples after the first
    failure won't even produce debugging output.
 
-   The doctest command line accepts the option ``-f`` as a shorthand for ``-o
-   FAIL_FAST``.
-
-   .. versionadded:: 3.4
-
 
 .. data:: REPORTING_FLAGS