Merge branch 'main' into fix-json-tool

Author: mpkocher

.github/CODEOWNERS

@@ -82,8 +82,8 @@ Tools/patchcheck/             @AA-Turner
 # Autotools
 configure*                       @erlend-aasland @corona10 @AA-Turner @emmatyping
 Makefile.pre.in                  @erlend-aasland @AA-Turner @emmatyping
+Modules/makesetup                @erlend-aasland @AA-Turner @emmatyping
 Modules/Setup*                   @erlend-aasland @AA-Turner @emmatyping
-Modules/makesetup                @emmatyping
 Tools/build/regen-configure.sh   @AA-Turner
 
 
@@ -406,11 +406,15 @@ Lib/test/test_dataclasses/    @ericvsmith
 
 # Dates and times
 Doc/**/*time.rst                 @pganssle @abalkin
+Doc/library/zoneinfo.rst         @pganssle
 Include/datetime.h               @pganssle @abalkin
 Include/internal/pycore_time.h   @pganssle @abalkin
+Lib/test/test_zoneinfo/          @pganssle
+Lib/zoneinfo/                    @pganssle
 Lib/*time.py                     @pganssle @abalkin
 Lib/test/datetimetester.py       @pganssle @abalkin
 Lib/test/test_*time.py           @pganssle @abalkin
+Modules/*zoneinfo*               @pganssle
 Modules/*time*                   @pganssle @abalkin
 Python/pytime.c                  @pganssle @abalkin
 

Doc/c-api/bytes.rst

@@ -47,6 +47,10 @@ called with a non-bytes parameter.
    *len* on success, and ``NULL`` on failure.  If *v* is ``NULL``, the contents of
    the bytes object are uninitialized.
 
+   .. deprecated:: 3.15
+      ``PyBytes_FromStringAndSize(NULL, len)`` is :term:`soft deprecated`,
+      use the :c:type:`PyBytesWriter` API instead.
+
 
 .. c:function:: PyObject* PyBytes_FromFormat(const char *format, ...)
 
@@ -219,3 +223,168 @@ called with a non-bytes parameter.
    reallocation fails, the original bytes object at *\*bytes* is deallocated,
    *\*bytes* is set to ``NULL``, :exc:`MemoryError` is set, and ``-1`` is
    returned.
+
+   .. deprecated:: 3.15
+      The function is :term:`soft deprecated`,
+      use the :c:type:`PyBytesWriter` API instead.
+
+.. _pybyteswriter:
+
+PyBytesWriter
+-------------
+
+The :c:type:`PyBytesWriter` API can be used to create a Python :class:`bytes`
+object.
+
+.. versionadded:: next
+
+.. c:type:: PyBytesWriter
+
+   A bytes writer instance.
+
+   The API is **not thread safe**: a writer should only be used by a single
+   thread at the same time.
+
+   The instance must be destroyed by :c:func:`PyBytesWriter_Finish` on
+   success, or :c:func:`PyBytesWriter_Discard` on error.
+
+
+Create, Finish, Discard
+^^^^^^^^^^^^^^^^^^^^^^^
+
+.. c:function:: PyBytesWriter* PyBytesWriter_Create(Py_ssize_t size)
+
+   Create a :c:type:`PyBytesWriter` to write *size* bytes.
+
+   If *size* is greater than zero, allocate *size* bytes, and set the
+   writer size to *size*. The caller is responsible to write *size*
+   bytes using :c:func:`PyBytesWriter_GetData`.
+
+   On error, set an exception and return NULL.
+
+   *size* must be positive or zero.
+
+.. c:function:: PyObject* PyBytesWriter_Finish(PyBytesWriter *writer)
+
+   Finish a :c:type:`PyBytesWriter` created by
+   :c:func:`PyBytesWriter_Create`.
+
+   On success, return a Python :class:`bytes` object.
+   On error, set an exception and return ``NULL``.
+
+   The writer instance is invalid after the call in any case.
+   No API can be called on the writer after :c:func:`PyBytesWriter_Finish`.
+
+.. c:function:: PyObject* PyBytesWriter_FinishWithSize(PyBytesWriter *writer, Py_ssize_t size)
+
+   Similar to :c:func:`PyBytesWriter_Finish`, but resize the writer
+   to *size* bytes before creating the :class:`bytes` object.
+
+.. c:function:: PyObject* PyBytesWriter_FinishWithPointer(PyBytesWriter *writer, void *buf)
+
+   Similar to :c:func:`PyBytesWriter_Finish`, but resize the writer
+   using *buf* pointer before creating the :class:`bytes` object.
+
+   Set an exception and return ``NULL`` if *buf* pointer is outside the
+   internal buffer bounds.
+
+   Function pseudo-code::
+
+       Py_ssize_t size = (char*)buf - (char*)PyBytesWriter_GetData(writer);
+       return PyBytesWriter_FinishWithSize(writer, size);
+
+.. c:function:: void PyBytesWriter_Discard(PyBytesWriter *writer)
+
+   Discard a :c:type:`PyBytesWriter` created by :c:func:`PyBytesWriter_Create`.
+
+   Do nothing if *writer* is ``NULL``.
+
+   The writer instance is invalid after the call.
+   No API can be called on the writer after :c:func:`PyBytesWriter_Discard`.
+
+High-level API
+^^^^^^^^^^^^^^
+
+.. c:function:: int PyBytesWriter_WriteBytes(PyBytesWriter *writer, const void *bytes, Py_ssize_t size)
+
+   Grow the *writer* internal buffer by *size* bytes,
+   write *size* bytes of *bytes* at the *writer* end,
+   and add *size* to the *writer* size.
+
+   If *size* is equal to ``-1``, call ``strlen(bytes)`` to get the
+   string length.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+.. c:function:: int PyBytesWriter_Format(PyBytesWriter *writer, const char *format, ...)
+
+   Similar to :c:func:`PyBytes_FromFormat`, but write the output directly at
+   the writer end. Grow the writer internal buffer on demand. Then add the
+   written size to the writer size.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+
+Getters
+^^^^^^^
+
+.. c:function:: Py_ssize_t PyBytesWriter_GetSize(PyBytesWriter *writer)
+
+   Get the writer size.
+
+.. c:function:: void* PyBytesWriter_GetData(PyBytesWriter *writer)
+
+   Get the writer data: start of the internal buffer.
+
+   The pointer is valid until :c:func:`PyBytesWriter_Finish` or
+   :c:func:`PyBytesWriter_Discard` is called on *writer*.
+
+
+Low-level API
+^^^^^^^^^^^^^
+
+.. c:function:: int PyBytesWriter_Resize(PyBytesWriter *writer, Py_ssize_t size)
+
+   Resize the writer to *size* bytes. It can be used to enlarge or to
+   shrink the writer.
+
+   Newly allocated bytes are left uninitialized.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+   *size* must be positive or zero.
+
+.. c:function:: int PyBytesWriter_Grow(PyBytesWriter *writer, Py_ssize_t grow)
+
+   Resize the writer by adding *grow* bytes to the current writer size.
+
+   Newly allocated bytes are left uninitialized.
+
+   On success, return ``0``.
+   On error, set an exception and return ``-1``.
+
+   *size* can be negative to shrink the writer.
+
+.. c:function:: void* PyBytesWriter_GrowAndUpdatePointer(PyBytesWriter *writer, Py_ssize_t size, void *buf)
+
+   Similar to :c:func:`PyBytesWriter_Grow`, but update also the *buf*
+   pointer.
+
+   The *buf* pointer is moved if the internal buffer is moved in memory.
+   The *buf* relative position within the internal buffer is left
+   unchanged.
+
+   On error, set an exception and return ``NULL``.
+
+   *buf* must not be ``NULL``.
+
+   Function pseudo-code::
+
+       Py_ssize_t pos = (char*)buf - (char*)PyBytesWriter_GetData(writer);
+       if (PyBytesWriter_Grow(writer, size) < 0) {
+           return NULL;
+       }
+       return (char*)PyBytesWriter_GetData(writer) + pos;

Doc/c-api/init.rst

@@ -41,7 +41,6 @@ The following functions can be safely called before Python is initialized:
   * :c:func:`PyObject_SetArenaAllocator`
   * :c:func:`Py_SetProgramName`
   * :c:func:`Py_SetPythonHome`
-  * :c:func:`PySys_ResetWarnOptions`
   * the configuration functions covered in :ref:`init-config`
 
 * Informative functions:
@@ -2011,6 +2010,11 @@ Reference tracing
    is set to :c:data:`PyRefTracer_DESTROY`). The **data** argument is the opaque pointer
    that was provided when :c:func:`PyRefTracer_SetTracer` was called.
 
+   If a new tracing function is registered replacing the current a call to the
+   trace function will be made with the object set to **NULL** and **event** set to
+   :c:data:`PyRefTracer_TRACKER_REMOVED`. This will happen just before the new
+   function is registered.
+
 .. versionadded:: 3.13
 
 .. c:var:: int PyRefTracer_CREATE
@@ -2023,6 +2027,13 @@ Reference tracing
    The value for the *event* parameter to :c:type:`PyRefTracer` functions when a Python
    object has been destroyed.
 
+.. c:var:: int PyRefTracer_TRACKER_REMOVED
+
+   The value for the *event* parameter to :c:type:`PyRefTracer` functions when the
+   current tracer is about to be replaced by a new one.
+
+   .. versionadded:: 3.14
+
 .. c:function:: int PyRefTracer_SetTracer(PyRefTracer tracer, void *data)
 
    Register a reference tracer function. The function will be called when a new
@@ -2038,6 +2049,10 @@ Reference tracing
 
    There must be an :term:`attached thread state` when calling this function.
 
+   If another tracer function was already registered, the old function will be
+   called with **event** set to :c:data:`PyRefTracer_TRACKER_REMOVED` just before
+   the new function is registered.
+
 .. versionadded:: 3.13
 
 .. c:function:: PyRefTracer PyRefTracer_GetTracer(void** data)

Doc/c-api/init_config.rst

@@ -1278,6 +1278,11 @@ PyConfig
 
       Default: ``0``.
 
+      .. deprecated-removed:: 3.15 3.17
+
+         The :option:`-b` and :option:`!-bb` options will become no-op in 3.17.
+         :c:member:`~PyConfig.bytes_warning` member will be removed in 3.17.
+
    .. c:member:: int warn_default_encoding
 
       If non-zero, emit a :exc:`EncodingWarning` warning when :class:`io.TextIOWrapper`

Doc/c-api/sys.rst

@@ -316,14 +316,6 @@ accessible to C code.  They all work with the current interpreter thread's
    case *name* is deleted from the sys module. Returns ``0`` on success, ``-1``
    on error.
 
-.. c:function:: void PySys_ResetWarnOptions()
-
-   Reset :data:`sys.warnoptions` to an empty list. This function may be
-   called prior to :c:func:`Py_Initialize`.
-
-   .. deprecated-removed:: 3.13 3.15
-      Clear :data:`sys.warnoptions` and :data:`!warnings.filters` instead.
-
 .. c:function:: void PySys_WriteStdout(const char *format, ...)
 
    Write the output string described by *format* to :data:`sys.stdout`.  No

Doc/c-api/veryhigh.rst

@@ -183,7 +183,8 @@ the same library that the Python runtime is using.
    objects *globals* and *locals* with the compiler flags specified by
    *flags*.  *globals* must be a dictionary; *locals* can be any object
    that implements the mapping protocol.  The parameter *start* specifies
-   the start token that should be used to parse the source code.
+   the start symbol and must one of the following:
+   :c:data:`Py_eval_input`, :c:data:`Py_file_input`, or :c:data:`Py_single_input`.
 
    Returns the result of executing the code as a Python object, or ``NULL`` if an
    exception was raised.
@@ -231,7 +232,7 @@ the same library that the Python runtime is using.
 .. c:function:: PyObject* Py_CompileStringObject(const char *str, PyObject *filename, int start, PyCompilerFlags *flags, int optimize)
 
    Parse and compile the Python source code in *str*, returning the resulting code
-   object.  The start token is given by *start*; this can be used to constrain the
+   object.  The start symbol is given by *start*; this can be used to constrain the
    code which can be compiled and should be :c:data:`Py_eval_input`,
    :c:data:`Py_file_input`, or :c:data:`Py_single_input`.  The filename specified by
    *filename* is used to construct the code object and may appear in tracebacks or

Doc/data/stable_abi.dat

@@ -59,7 +59,7 @@ Pending removal in Python 3.15
     Set :c:member:`PyConfig.program_name` instead.
   * :c:func:`!Py_SetPythonHome()`:
     Set :c:member:`PyConfig.home` instead.
-  * :c:func:`PySys_ResetWarnOptions`:
+  * :c:func:`!PySys_ResetWarnOptions`:
     Clear :data:`sys.warnoptions` and :data:`!warnings.filters` instead.
 
   The :c:func:`Py_InitializeFromConfig` API should be used with

Doc/deprecations/c-api-pending-removal-in-3.15.rst

@@ -1,11 +1,12 @@
 Pending removal in Python 3.18
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-* Deprecated private functions (:gh:`128863`):
+* The following private functions are deprecated
+  and planned for removal in Python 3.18:
 
   * :c:func:`!_PyBytes_Join`: use :c:func:`PyBytes_Join`.
   * :c:func:`!_PyDict_GetItemStringWithError`: use :c:func:`PyDict_GetItemStringRef`.
-  * :c:func:`!_PyDict_Pop()`: :c:func:`PyDict_Pop`.
+  * :c:func:`!_PyDict_Pop()`: use :c:func:`PyDict_Pop`.
   * :c:func:`!_PyLong_Sign()`: use :c:func:`PyLong_GetSign`.
   * :c:func:`!_PyLong_FromDigits` and :c:func:`!_PyLong_New`:
     use :c:func:`PyLongWriter_Create`.
@@ -31,7 +32,7 @@ Pending removal in Python 3.18
     :c:func:`PyUnicodeWriter_WriteSubstring(writer, str, start, end) <PyUnicodeWriter_WriteSubstring>`.
   * :c:func:`!_PyUnicodeWriter_WriteASCIIString`:
     replace ``_PyUnicodeWriter_WriteASCIIString(&writer, str)`` with
-    :c:func:`PyUnicodeWriter_WriteUTF8(writer, str) <PyUnicodeWriter_WriteUTF8>`.
+    :c:func:`PyUnicodeWriter_WriteASCII(writer, str) <PyUnicodeWriter_WriteASCII>`.
   * :c:func:`!_PyUnicodeWriter_WriteLatin1String`:
     replace ``_PyUnicodeWriter_WriteLatin1String(&writer, str)`` with
     :c:func:`PyUnicodeWriter_WriteUTF8(writer, str) <PyUnicodeWriter_WriteUTF8>`.
@@ -41,5 +42,6 @@ Pending removal in Python 3.18
   * :c:func:`!_Py_fopen_obj`: use :c:func:`Py_fopen`.
 
   The `pythoncapi-compat project
-  <https://github.com/python/pythoncapi-compat/>`__ can be used to get these
-  new public functions on Python 3.13 and older.
+  <https://github.com/python/pythoncapi-compat/>`__ can be used to get
+  these new public functions on Python 3.13 and older.
+  (Contributed by Victor Stinner in :gh:`128863`.)

Doc/deprecations/c-api-pending-removal-in-3.18.rst

@@ -38,12 +38,6 @@ Pending removal in Python 3.14
     is no current event loop set and it decides to create one.
     (Contributed by Serhiy Storchaka and Guido van Rossum in :gh:`100160`.)
 
-* :mod:`collections.abc`: Deprecated :class:`!collections.abc.ByteString`.
-  Prefer :class:`!Sequence` or :class:`~collections.abc.Buffer`.
-  For use in typing, prefer a union, like ``bytes | bytearray``,
-  or :class:`collections.abc.Buffer`.
-  (Contributed by Shantanu Jain in :gh:`91896`.)
-
 * :mod:`email`: Deprecated the *isdst* parameter in :func:`email.utils.localtime`.
   (Contributed by Alan Williams in :gh:`72346`.)
 
@@ -96,9 +90,6 @@ Pending removal in Python 3.14
     if :ref:`named placeholders <sqlite3-placeholders>` are used and
     *parameters* is a sequence instead of a :class:`dict`.
 
-* :mod:`typing`: :class:`!typing.ByteString`, deprecated since Python 3.9,
-  now causes a :exc:`DeprecationWarning` to be emitted when it is used.
-
 * :mod:`urllib`:
   :class:`!urllib.parse.Quoter` is deprecated: it was not intended to be a
   public API.