From dd86fb4ba5a9db1c1e7293917af68d0cf0ddeaaf Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Date: Wed, 3 Sep 2025 07:20:16 +0100 Subject: [PATCH 1/6] GH-101100: Resolve reference warnings in whatsnew/3.7.rst (#138410) Resolve reference warnings in whatsnew/3.7.rst --- Doc/howto/instrumentation.rst | 2 + Doc/library/hmac.rst | 4 +- Doc/reference/datamodel.rst | 5 ++ Doc/tools/.nitignore | 1 - Doc/whatsnew/3.7.rst | 95 +++++++++++++++++++---------------- 5 files changed, 61 insertions(+), 46 deletions(-) diff --git a/Doc/howto/instrumentation.rst b/Doc/howto/instrumentation.rst index 6e03ef20a21fa3..b3db1189e5dcbc 100644 --- a/Doc/howto/instrumentation.rst +++ b/Doc/howto/instrumentation.rst @@ -269,6 +269,8 @@ should instead read: (assuming a :ref:`debug build ` of CPython 3.6) +.. _static-markers: + Available static markers ------------------------ diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst index 57076c38086c79..d5608bd7543eb1 100644 --- a/Doc/library/hmac.rst +++ b/Doc/library/hmac.rst @@ -50,7 +50,9 @@ cannot be used with HMAC. .. versionadded:: 3.7 -An HMAC object has the following methods: +.. class:: HMAC + + An HMAC object has the following methods: .. method:: HMAC.update(msg) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 7af3457070b84a..da04cfde3bd587 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2351,6 +2351,9 @@ Customizing module attribute access single: __dir__ (module attribute) single: __class__ (module attribute) +.. method:: module.__getattr__ + module.__dir__ + Special names ``__getattr__`` and ``__dir__`` can be also used to customize access to module attributes. The ``__getattr__`` function at the module level should accept one argument which is the name of an attribute and return the @@ -2364,6 +2367,8 @@ The ``__dir__`` function should accept no arguments, and return an iterable of strings that represents the names accessible on module. If present, this function overrides the standard :func:`dir` search on a module. +.. attribute:: module.__class__ + For a more fine grained customization of the module behavior (setting attributes, properties, etc.), one can set the ``__class__`` attribute of a module object to a subclass of :class:`types.ModuleType`. For example:: diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 7d0512fd0fa72d..07671fe17c1d89 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -66,6 +66,5 @@ Doc/whatsnew/3.3.rst Doc/whatsnew/3.4.rst Doc/whatsnew/3.5.rst Doc/whatsnew/3.6.rst -Doc/whatsnew/3.7.rst Doc/whatsnew/3.8.rst Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index f420fa5c04479b..6e6934befccb3b 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -320,9 +320,9 @@ effort will be made to add such support. PEP 562: Customization of Access to Module Attributes ----------------------------------------------------- -Python 3.7 allows defining :meth:`__getattr__` on modules and will call +Python 3.7 allows defining :meth:`~module.__getattr__` on modules and will call it whenever a module attribute is otherwise not found. Defining -:meth:`__dir__` on modules is now also allowed. +:meth:`~module.__dir__` on modules is now also allowed. A typical example of where this may be useful is module attribute deprecation and lazy loading. @@ -409,8 +409,8 @@ PEP 560: Core Support for ``typing`` module and Generic Types Initially :pep:`484` was designed in such way that it would not introduce *any* changes to the core CPython interpreter. Now type hints and the :mod:`typing` module are extensively used by the community, so this restriction is removed. -The PEP introduces two special methods :meth:`__class_getitem__` and -``__mro_entries__``, these methods are now used by most classes and special +The PEP introduces two special methods :meth:`~object.__class_getitem__` and +:meth:`~object.__mro_entries__`, these methods are now used by most classes and special constructs in :mod:`typing`. As a result, the speed of various operations with types increased up to 7 times, the generic types can be used without metaclass conflicts, and several long standing bugs in :mod:`typing` module are @@ -603,7 +603,7 @@ The new :mod:`importlib.resources` module provides several new APIs and one new ABC for access to, opening, and reading *resources* inside packages. Resources are roughly similar to files inside packages, but they needn't be actual files on the physical file system. Module loaders can provide a -:meth:`get_resource_reader` function which returns +:meth:`!get_resource_reader` function which returns a :class:`importlib.abc.ResourceReader` instance to support this new API. Built-in file path loaders and zip file loaders both support this. @@ -910,9 +910,9 @@ which allows listing the names of properties which should not become enum members. (Contributed by Ethan Furman in :issue:`31801`.) -In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +In Python 3.8, attempting to check for non-Enum objects in :class:`~enum.Enum` classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, -attempting to check for non-Flag objects in a :class:`Flag` member will +attempting to check for non-Flag objects in a :class:`~enum.Flag` member will raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations return :const:`False` instead and are deprecated. (Contributed by Ethan Furman in :issue:`33217`.) @@ -969,7 +969,7 @@ uses the current working directory. (Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) The new :class:`ThreadingHTTPServer ` class -uses threads to handle requests using :class:`~socketserver.ThreadingMixin`. +uses threads to handle requests using :class:`~socketserver.ThreadingMixIn`. It is used when ``http.server`` is run with ``-m``. (Contributed by Julien Palard in :issue:`31639`.) @@ -1052,12 +1052,12 @@ support the loading of resources from packages. See also lacks a spec. (Contributed by Garvit Khatri in :issue:`29851`.) -:func:`importlib.find_spec` now raises :exc:`ModuleNotFoundError` instead of +:func:`importlib.util.find_spec` now raises :exc:`ModuleNotFoundError` instead of :exc:`AttributeError` if the specified parent module is not a package (i.e. lacks a ``__path__`` attribute). (Contributed by Milan Oberkirch in :issue:`30436`.) -The new :func:`importlib.source_hash` can be used to compute the hash of +The new :func:`importlib.util.source_hash` can be used to compute the hash of the passed source. A :ref:`hash-based .pyc file ` embeds the value returned by this function. @@ -1148,7 +1148,7 @@ running. (Contributed by Antoine Pitrou in :issue:`30596`.) The new :meth:`Process.kill() ` method can -be used to terminate the process using the :data:`SIGKILL` signal on Unix. +be used to terminate the process using the :data:`~signal.SIGKILL` signal on Unix. (Contributed by Vitor Pereira in :issue:`30794`.) Non-daemonic threads created by :class:`~multiprocessing.Process` are now @@ -1280,9 +1280,10 @@ This function should be used instead of :func:`os.close` for better compatibility across platforms. (Contributed by Christian Heimes in :issue:`32454`.) -The :mod:`socket` module now exposes the :const:`socket.TCP_CONGESTION` -(Linux 2.6.13), :const:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and -:const:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. +The :mod:`socket` module now exposes the :ref:`socket.TCP_CONGESTION +` (Linux 2.6.13), :ref:`socket.TCP_USER_TIMEOUT +` (Linux 2.6.37), and :ref:`socket.TCP_NOTSENT_LOWAT +` (Linux 3.12) constants. (Contributed by Omar Sandoval in :issue:`26273` and Nathaniel J. Smith in :issue:`29728`.) @@ -1298,11 +1299,14 @@ by default. socketserver ------------ -:meth:`socketserver.ThreadingMixIn.server_close` now waits until all non-daemon -threads complete. :meth:`socketserver.ForkingMixIn.server_close` now waits +:meth:`socketserver.ThreadingMixIn.server_close +` now waits until all non-daemon +threads complete. :meth:`socketserver.ForkingMixIn.server_close +` now waits until all child processes complete. -Add a new :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to +Add a new :attr:`socketserver.ForkingMixIn.block_on_close +` class attribute to :class:`socketserver.ForkingMixIn` and :class:`socketserver.ThreadingMixIn` classes. Set the class attribute to ``False`` to get the pre-3.7 behaviour. @@ -1323,7 +1327,7 @@ ssl --- The :mod:`ssl` module now uses OpenSSL's builtin API instead of -:func:`~ssl.match_hostname` to check a host name or an IP address. Values +:func:`!match_hostname` to check a host name or an IP address. Values are validated during TLS handshake. Any certificate validation error including failing the host name check now raises :exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper @@ -1341,7 +1345,7 @@ Host name validation can be customized with The ``ssl`` module no longer sends IP addresses in SNI TLS extension. (Contributed by Christian Heimes in :issue:`32185`.) -:func:`~ssl.match_hostname` no longer supports partial wildcards like +:func:`!match_hostname` no longer supports partial wildcards like ``www*.example.org``. (Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in :issue:`31399`.) @@ -1438,7 +1442,7 @@ The new :func:`sys.get_coroutine_origin_tracking_depth` function returns the current coroutine origin tracking depth, as set by the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio` has been converted to use this new API instead of -the deprecated :func:`sys.set_coroutine_wrapper`. +the deprecated :func:`!sys.set_coroutine_wrapper`. (Contributed by Nathaniel J. Smith in :issue:`32591`.) @@ -1615,7 +1619,7 @@ external entities by default. xml.etree --------- -:ref:`ElementPath ` predicates in the :meth:`find` +:ref:`ElementPath ` predicates in the :meth:`!find` methods can now compare text of the current node with ``[. = "text"]``, not only text in children. Predicates also allow adding spaces for better readability. (Contributed by Stefan Behnel in :issue:`31648`.) @@ -1624,7 +1628,7 @@ better readability. (Contributed by Stefan Behnel in :issue:`31648`.) xmlrpc.server ------------- -:meth:`SimpleXMLRPCDispatcher.register_function ` +:meth:`!SimpleXMLRPCDispatcher.register_function` can now be used as a decorator. (Contributed by Xiang Zhang in :issue:`7769`.) @@ -1682,15 +1686,15 @@ The :mod:`tracemalloc` now exposes a C API through the new functions. (Contributed by Victor Stinner in :issue:`30054`.) -The new :c:func:`import__find__load__start` and -:c:func:`import__find__load__done` static markers can be used to trace -module imports. +The new :ref:`import__find__load__start ` and +:ref:`import__find__load__done ` static markers can be used +to trace module imports. (Contributed by Christian Heimes in :issue:`31574`.) The fields :c:member:`!name` and :c:member:`!doc` of structures :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, -and :c:struct:`wrapperbase` are now of type ``const char *`` rather of +and :c:struct:`!wrapperbase` are now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` @@ -1719,8 +1723,8 @@ Added C API support for timezones with timezone constructors and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. Contributed by Paul Ganssle in :issue:`10381`. -The type of results of :c:func:`PyThread_start_new_thread` and -:c:func:`PyThread_get_thread_ident`, and the *id* parameter of +The type of results of :c:func:`!PyThread_start_new_thread` and +:c:func:`!PyThread_get_thread_ident`, and the *id* parameter of :c:func:`PyThreadState_SetAsyncExc` changed from :c:expr:`long` to :c:expr:`unsigned long`. (Contributed by Serhiy Storchaka in :issue:`6532`.) @@ -1847,8 +1851,8 @@ make the creation of named tuples 4 to 6 times faster. (Contributed by Jelle Zijlstra with further improvements by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) -:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to -30% faster in the common case. +:meth:`datetime.date.fromordinal` and :meth:`datetime.date.fromtimestamp` +are now up to 30% faster in the common case. (Contributed by Paul Ganssle in :issue:`32403`.) The :func:`os.fwalk` function is now up to 2 times faster thanks to @@ -1997,9 +2001,9 @@ modes (this will be an error in future Python releases). enum ---- -In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +In Python 3.8, attempting to check for non-Enum objects in :class:`~enum.Enum` classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, -attempting to check for non-Flag objects in a :class:`Flag` member will +attempting to check for non-Flag objects in a :class:`~enum.Flag` member will raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations return :const:`False` instead. (Contributed by Ethan Furman in :issue:`33217`.) @@ -2034,14 +2038,14 @@ favour of :class:`importlib.abc.ResourceReader`. locale ------ -:func:`locale.format` has been deprecated, use :meth:`locale.format_string` +:func:`!locale.format` has been deprecated, use :meth:`locale.format_string` instead. (Contributed by Garvit in :issue:`10379`.) macpath ------- -The :mod:`macpath` is now deprecated and will be removed in Python 3.8. +The :mod:`!macpath` is now deprecated and will be removed in Python 3.8. (Contributed by Chi Hsuan Yen in :issue:`9850`.) @@ -2066,7 +2070,7 @@ if the passed argument is larger than 16 bits, an exception will be raised. ssl --- -:func:`ssl.wrap_socket` is deprecated. Use +:func:`!ssl.wrap_socket` is deprecated. Use :meth:`ssl.SSLContext.wrap_socket` instead. (Contributed by Christian Heimes in :issue:`28124`.) @@ -2082,8 +2086,8 @@ Use :func:`!sunau.open` instead. sys --- -Deprecated :func:`sys.set_coroutine_wrapper` and -:func:`sys.get_coroutine_wrapper`. +Deprecated :func:`!sys.set_coroutine_wrapper` and +:func:`!sys.get_coroutine_wrapper`. The undocumented ``sys.callstats()`` function has been deprecated and will be removed in a future Python version. @@ -2093,7 +2097,7 @@ will be removed in a future Python version. wave ---- -:func:`wave.openfp` has been deprecated and will be removed in Python 3.9. +:func:`!wave.openfp` has been deprecated and will be removed in Python 3.9. Use :func:`wave.open` instead. (Contributed by Brian Curtin in :issue:`31985`.) @@ -2173,8 +2177,8 @@ The following features and APIs have been removed from Python 3.7: * Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and ``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result - of functions :func:`~plistlib.readPlist` and - :func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer + of functions :func:`!readPlist` and + :func:`!readPlistFromBytes` are now normal dicts. You no longer can use attribute access to access items of these dictionaries. * The ``asyncio.windows_utils.socketpair()`` function has been @@ -2191,7 +2195,7 @@ The following features and APIs have been removed from Python 3.7: * Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` objects is now prohibited. The constructors were never documented, tested, or designed as public constructors. Users were supposed to use - :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. + :func:`!ssl.wrap_socket` or :class:`ssl.SSLContext`. (Contributed by Christian Heimes in :issue:`32951`.) * The unused ``distutils`` ``install_misc`` command has been removed. @@ -2275,15 +2279,18 @@ Changes in Python Behavior Changes in the Python API ------------------------- -* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all +* :meth:`socketserver.ThreadingMixIn.server_close + ` now waits until all non-daemon threads complete. Set the new :attr:`socketserver.ThreadingMixIn.block_on_close` class attribute to ``False`` to get the pre-3.7 behaviour. (Contributed by Victor Stinner in :issue:`31233` and :issue:`33540`.) -* :meth:`socketserver.ForkingMixIn.server_close` now waits until all +* :meth:`socketserver.ForkingMixIn.server_close + ` now waits until all child processes complete. Set the new - :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to ``False`` + :attr:`socketserver.ForkingMixIn.block_on_close + ` class attribute to ``False`` to get the pre-3.7 behaviour. (Contributed by Victor Stinner in :issue:`31151` and :issue:`33540`.) From 51244ba16a7312945a60dff18ed155a581295fbf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Wed, 3 Sep 2025 09:55:53 +0200 Subject: [PATCH 2/6] gh-116946: add `Py_TPFLAGS_IMMUTABLETYPE` to `_random.Random` (#138341) --- .../2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst | 2 ++ Modules/_randommodule.c | 13 ++++++++----- 2 files changed, 10 insertions(+), 5 deletions(-) create mode 100644 Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst diff --git a/Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst b/Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst new file mode 100644 index 00000000000000..015cf24c8869f8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2025-09-02-10-23-09.gh-issue-116946.U6RpwK.rst @@ -0,0 +1,2 @@ +The :class:`!_random.Random` C type is now immutable. Patch by Bénédikt +Tran. diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c index 2f4f388ce1161a..aa2fd28c232f28 100644 --- a/Modules/_randommodule.c +++ b/Modules/_randommodule.c @@ -595,11 +595,14 @@ static PyType_Slot Random_Type_slots[] = { }; static PyType_Spec Random_Type_spec = { - "_random.Random", - sizeof(RandomObject), - 0, - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, - Random_Type_slots + .name = "_random.Random", + .basicsize = sizeof(RandomObject), + .flags = ( + Py_TPFLAGS_DEFAULT + | Py_TPFLAGS_BASETYPE + | Py_TPFLAGS_IMMUTABLETYPE + ), + .slots = Random_Type_slots }; PyDoc_STRVAR(module_doc, From 572df47840d910b9fc9cd951074232ae89442be1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Wed, 3 Sep 2025 10:17:17 +0200 Subject: [PATCH 3/6] gh-116946: fully implement GC protocol for `_curses_panel.panel` (#138333) This commit fixes possible reference loops via `panel.set_userptr` by implementing `tp_clear` and `tp_traverse` for panel objects. --- Modules/_curses_panel.c | 58 +++++++++++++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 14 deletions(-) diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c index 3408a505575580..66a8c40953da8c 100644 --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -410,8 +410,11 @@ static PyObject * PyCursesPanel_New(_curses_panel_state *state, PANEL *pan, PyCursesWindowObject *wo) { - PyCursesPanelObject *po = PyObject_New(PyCursesPanelObject, - state->PyCursesPanel_Type); + assert(state != NULL); + PyTypeObject *type = state->PyCursesPanel_Type; + assert(type != NULL); + assert(type->tp_alloc != NULL); + PyCursesPanelObject *po = (PyCursesPanelObject *)type->tp_alloc(type, 0); if (po == NULL) { return NULL; } @@ -426,20 +429,31 @@ PyCursesPanel_New(_curses_panel_state *state, PANEL *pan, return (PyObject *)po; } +static int +PyCursesPanel_Clear(PyObject *op) +{ + PyCursesPanelObject *self = _PyCursesPanelObject_CAST(op); + PyObject *extra = (PyObject *)panel_userptr(self->pan); + if (extra != NULL) { + Py_DECREF(extra); + if (set_panel_userptr(self->pan, NULL) == ERR) { + curses_panel_panel_set_error(self, "set_panel_userptr", NULL); + return -1; + } + } + // self->wo should not be cleared because an associated WINDOW may exist + return 0; +} + static void PyCursesPanel_Dealloc(PyObject *self) { - PyObject *tp, *obj; - PyCursesPanelObject *po = _PyCursesPanelObject_CAST(self); + PyTypeObject *tp = Py_TYPE(self); + PyObject_GC_UnTrack(self); - tp = (PyObject *) Py_TYPE(po); - obj = (PyObject *) panel_userptr(po->pan); - if (obj) { - Py_DECREF(obj); - if (set_panel_userptr(po->pan, NULL) == ERR) { - curses_panel_panel_set_error(po, "set_panel_userptr", "__del__"); - PyErr_FormatUnraisable("Exception ignored in PyCursesPanel_Dealloc()"); - } + PyCursesPanelObject *po = _PyCursesPanelObject_CAST(self); + if (PyCursesPanel_Clear(self) < 0) { + PyErr_FormatUnraisable("Exception ignored in PyCursesPanel_Dealloc()"); } if (del_panel(po->pan) == ERR && !PyErr_Occurred()) { curses_panel_panel_set_error(po, "del_panel", "__del__"); @@ -452,10 +466,20 @@ PyCursesPanel_Dealloc(PyObject *self) PyErr_FormatUnraisable("Exception ignored in PyCursesPanel_Dealloc()"); } } - PyObject_Free(po); + tp->tp_free(po); Py_DECREF(tp); } +static int +PyCursesPanel_Traverse(PyObject *op, visitproc visit, void *arg) +{ + PyCursesPanelObject *self = _PyCursesPanelObject_CAST(op); + Py_VISIT(Py_TYPE(op)); + Py_VISIT(panel_userptr(self->pan)); + Py_VISIT(self->wo); + return 0; +} + /* panel_above(NULL) returns the bottom panel in the stack. To get this behaviour we use curses.panel.bottom_panel(). */ /*[clinic input] @@ -647,7 +671,9 @@ static PyMethodDef PyCursesPanel_Methods[] = { /* -------------------------------------------------------*/ static PyType_Slot PyCursesPanel_Type_slots[] = { + {Py_tp_clear, PyCursesPanel_Clear}, {Py_tp_dealloc, PyCursesPanel_Dealloc}, + {Py_tp_traverse, PyCursesPanel_Traverse}, {Py_tp_methods, PyCursesPanel_Methods}, {0, 0}, }; @@ -655,7 +681,11 @@ static PyType_Slot PyCursesPanel_Type_slots[] = { static PyType_Spec PyCursesPanel_Type_spec = { .name = "_curses_panel.panel", .basicsize = sizeof(PyCursesPanelObject), - .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_DISALLOW_INSTANTIATION, + .flags = ( + Py_TPFLAGS_DEFAULT + | Py_TPFLAGS_DISALLOW_INSTANTIATION + | Py_TPFLAGS_HAVE_GC + ), .slots = PyCursesPanel_Type_slots }; From 4a33077fdb546db71b16d50cdd552529a9a1c910 Mon Sep 17 00:00:00 2001 From: Serhiy Storchaka Date: Wed, 3 Sep 2025 11:26:56 +0300 Subject: [PATCH 4/6] gh-138264: Fix gcc 14 compiler warnings (GH-138265) --- Include/internal/pycore_stats.h | 2 +- Lib/test/test_generated_cases.py | 2 +- Modules/_sre/sre.c | 2 +- Python/executor_cases.c.h | 8 +++--- Python/generated_cases.c.h | 30 +++++++++++----------- Tools/cases_generator/generators_common.py | 2 +- 6 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Include/internal/pycore_stats.h b/Include/internal/pycore_stats.h index ab649574f33dbf..24f239a2135b93 100644 --- a/Include/internal/pycore_stats.h +++ b/Include/internal/pycore_stats.h @@ -77,7 +77,7 @@ PyAPI_FUNC(PyObject*) _Py_GetSpecializationStats(void); #define RARE_EVENT_INTERP_INC(interp, name) \ do { \ /* saturating add */ \ - int val = FT_ATOMIC_LOAD_UINT8_RELAXED(interp->rare_events.name); \ + uint8_t val = FT_ATOMIC_LOAD_UINT8_RELAXED(interp->rare_events.name); \ if (val < UINT8_MAX) { \ FT_ATOMIC_STORE_UINT8(interp->rare_events.name, val + 1); \ } \ diff --git a/Lib/test/test_generated_cases.py b/Lib/test/test_generated_cases.py index ec44a0f9ce3fb3..6e0e2eafd3281c 100644 --- a/Lib/test/test_generated_cases.py +++ b/Lib/test/test_generated_cases.py @@ -1572,7 +1572,7 @@ def test_instruction_size_macro(self): frame->instr_ptr = next_instr; next_instr += 1; INSTRUCTION_STATS(OP); - frame->return_offset = 1 ; + frame->return_offset = 1u ; DISPATCH(); } """ diff --git a/Modules/_sre/sre.c b/Modules/_sre/sre.c index 49eb52b635bfbb..fdf00e6499cb6b 100644 --- a/Modules/_sre/sre.c +++ b/Modules/_sre/sre.c @@ -2359,7 +2359,7 @@ match_getindex(MatchObject* self, PyObject* index) } // Check that i*2 cannot overflow to make static analyzers happy - assert(i <= SRE_MAXGROUPS); + assert((size_t)i <= SRE_MAXGROUPS); return i; } diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 3dcb2decc43737..635ca659394c32 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -1688,7 +1688,7 @@ _PyInterpreterFrame* pushed_frame = _PyFrame_PushUnchecked(tstate, getitem, 2, frame); pushed_frame->localsplus[0] = container; pushed_frame->localsplus[1] = sub; - frame->return_offset = 6 ; + frame->return_offset = 6u ; new_frame = PyStackRef_Wrap(pushed_frame); stack_pointer[-3] = new_frame; stack_pointer += -2; @@ -2057,8 +2057,8 @@ gen->gi_frame_state = FRAME_EXECUTING; gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; - assert( 2 + oparg <= UINT16_MAX); - frame->return_offset = (uint16_t)( 2 + oparg); + assert( 2u + oparg <= UINT16_MAX); + frame->return_offset = (uint16_t)( 2u + oparg); pushed_frame->previous = frame; gen_frame = PyStackRef_Wrap(pushed_frame); stack_pointer[-1] = gen_frame; @@ -4610,7 +4610,7 @@ gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; pushed_frame->previous = frame; - frame->return_offset = (uint16_t)( 2 + oparg); + frame->return_offset = (uint16_t)( 2u + oparg); gen_frame = PyStackRef_Wrap(pushed_frame); stack_pointer[0] = gen_frame; stack_pointer += 1; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 7547eaad125370..c3e37f080fa6e4 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -653,7 +653,7 @@ _PyInterpreterFrame* pushed_frame = _PyFrame_PushUnchecked(tstate, getitem, 2, frame); pushed_frame->localsplus[0] = container; pushed_frame->localsplus[1] = sub; - frame->return_offset = 6 ; + frame->return_offset = 6u ; new_frame = PyStackRef_Wrap(pushed_frame); } // _PUSH_FRAME @@ -1587,7 +1587,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - frame->return_offset = 4 ; + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -2603,7 +2603,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 1 == 1); + assert( 1u == 1); frame->return_offset = 1; DISPATCH_INLINED(new_frame); } @@ -2883,8 +2883,8 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 4 == 1 + INLINE_CACHE_ENTRIES_CALL_KW); - frame->return_offset = 4 ; + assert( 4u == 1 + INLINE_CACHE_ENTRIES_CALL_KW); + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -5716,7 +5716,7 @@ gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; pushed_frame->previous = frame; - frame->return_offset = (uint16_t)( 2 + oparg); + frame->return_offset = (uint16_t)( 2u + oparg); gen_frame = PyStackRef_Wrap(pushed_frame); } // _PUSH_FRAME @@ -6334,7 +6334,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - frame->return_offset = 4 ; + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -6547,7 +6547,7 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 1 == 1); + assert( 1u == 1); frame->return_offset = 1; DISPATCH_INLINED(new_frame); } @@ -6690,8 +6690,8 @@ if (new_frame == NULL) { JUMP_TO_LABEL(error); } - assert( 4 == 1 + INLINE_CACHE_ENTRIES_CALL_KW); - frame->return_offset = 4 ; + assert( 4u == 1 + INLINE_CACHE_ENTRIES_CALL_KW); + frame->return_offset = 4u ; DISPATCH_INLINED(new_frame); } STACKREFS_TO_PYOBJECTS(arguments, total_args, args_o); @@ -8060,7 +8060,7 @@ stack_pointer += -1; assert(WITHIN_STACK_BOUNDS()); new_frame->localsplus[1] = PyStackRef_FromPyObjectNew(name); - frame->return_offset = 10 ; + frame->return_offset = 10u ; DISPATCH_INLINED(new_frame); } @@ -10459,8 +10459,8 @@ gen->gi_frame_state = FRAME_EXECUTING; gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; - assert( 2 + oparg <= UINT16_MAX); - frame->return_offset = (uint16_t)( 2 + oparg); + assert( 2u + oparg <= UINT16_MAX); + frame->return_offset = (uint16_t)( 2u + oparg); assert(gen_frame->previous == NULL); gen_frame->previous = frame; DISPATCH_INLINED(gen_frame); @@ -10560,8 +10560,8 @@ gen->gi_frame_state = FRAME_EXECUTING; gen->gi_exc_state.previous_item = tstate->exc_info; tstate->exc_info = &gen->gi_exc_state; - assert( 2 + oparg <= UINT16_MAX); - frame->return_offset = (uint16_t)( 2 + oparg); + assert( 2u + oparg <= UINT16_MAX); + frame->return_offset = (uint16_t)( 2u + oparg); pushed_frame->previous = frame; gen_frame = PyStackRef_Wrap(pushed_frame); } diff --git a/Tools/cases_generator/generators_common.py b/Tools/cases_generator/generators_common.py index cc0edcdf6006fd..61e855eb003706 100644 --- a/Tools/cases_generator/generators_common.py +++ b/Tools/cases_generator/generators_common.py @@ -470,7 +470,7 @@ def instruction_size(self, """Replace the INSTRUCTION_SIZE macro with the size of the current instruction.""" if uop.instruction_size is None: raise analysis_error("The INSTRUCTION_SIZE macro requires uop.instruction_size to be set", tkn) - self.out.emit(f" {uop.instruction_size} ") + self.out.emit(f" {uop.instruction_size}u ") return True def _print_storage(self, reason:str, storage: Storage) -> None: From 7274d076077212ecda4e83932dcc4ae69c62af58 Mon Sep 17 00:00:00 2001 From: Serhiy Storchaka Date: Wed, 3 Sep 2025 11:31:34 +0300 Subject: [PATCH 5/6] gh-78502: Add a trackfd parameter to mmap.mmap() on Windows (GH-138238) If trackfd is False, the file handle corresponding to fileno will not be duplicated. --- Doc/library/mmap.rst | 16 +++- Doc/whatsnew/3.15.rst | 9 +++ Lib/test/test_mmap.py | 77 +++++++++---------- ...5-08-29-12-05-33.gh-issue-78502.VpIMxg.rst | 2 + Modules/mmapmodule.c | 59 +++++++------- 5 files changed, 92 insertions(+), 71 deletions(-) create mode 100644 Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst diff --git a/Doc/library/mmap.rst b/Doc/library/mmap.rst index 5d81477443ca31..d9d401a2789c0e 100644 --- a/Doc/library/mmap.rst +++ b/Doc/library/mmap.rst @@ -48,10 +48,11 @@ update the underlying file. To map anonymous memory, -1 should be passed as the fileno along with the length. -.. class:: mmap(fileno, length, tagname=None, access=ACCESS_DEFAULT, offset=0) +.. class:: mmap(fileno, length, tagname=None, \ + access=ACCESS_DEFAULT, offset=0, *, trackfd=True) **(Windows version)** Maps *length* bytes from the file specified by the - file handle *fileno*, and creates a mmap object. If *length* is larger + file descriptor *fileno*, and creates a mmap object. If *length* is larger than the current size of the file, the file is extended to contain *length* bytes. If *length* is ``0``, the maximum length of the map is the current size of the file, except that if the file is empty Windows raises an @@ -69,6 +70,17 @@ To map anonymous memory, -1 should be passed as the fileno along with the length will be relative to the offset from the beginning of the file. *offset* defaults to 0. *offset* must be a multiple of the :const:`ALLOCATIONGRANULARITY`. + If *trackfd* is ``False``, the file handle corresponding to *fileno* will + not be duplicated, and the resulting :class:`!mmap` object will not + be associated with the map's underlying file. + This means that the :meth:`~mmap.mmap.size` and :meth:`~mmap.mmap.resize` + methods will fail. + This mode is useful to limit the number of open file handles. + The original file can be renamed (but not deleted) after closing *fileno*. + + .. versionchanged:: next + The *trackfd* parameter was added. + .. audit-event:: mmap.__new__ fileno,length,access,offset mmap.mmap .. class:: mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, \ diff --git a/Doc/whatsnew/3.15.rst b/Doc/whatsnew/3.15.rst index b5e138aa674697..932bb100cbee23 100644 --- a/Doc/whatsnew/3.15.rst +++ b/Doc/whatsnew/3.15.rst @@ -358,6 +358,15 @@ math (Contributed by Bénédikt Tran in :gh:`135853`.) +mmap +---- + +* :class:`mmap.mmap` now has a *trackfd* parameter on Windows; + if it is ``False``, the file handle corresponding to *fileno* will + not be duplicated. + (Contributed by Serhiy Storchaka in :gh:`78502`.) + + os.path ------- diff --git a/Lib/test/test_mmap.py b/Lib/test/test_mmap.py index 32881d36dcae10..da69770915092a 100644 --- a/Lib/test/test_mmap.py +++ b/Lib/test/test_mmap.py @@ -1,3 +1,4 @@ +from test import support from test.support import ( requires, _2G, _4G, gc_collect, cpython_only, is_emscripten, is_apple, in_systemd_nspawn_sync_suppressed, @@ -269,41 +270,44 @@ def test_access_parameter(self): self.assertRaises(TypeError, m.write_byte, 0) m.close() - @unittest.skipIf(os.name == 'nt', 'trackfd not present on Windows') - def test_trackfd_parameter(self): + @support.subTests('close_original_fd', (True, False)) + def test_trackfd_parameter(self, close_original_fd): size = 64 with open(TESTFN, "wb") as f: f.write(b"a"*size) - for close_original_fd in True, False: - with self.subTest(close_original_fd=close_original_fd): - with open(TESTFN, "r+b") as f: - with mmap.mmap(f.fileno(), size, trackfd=False) as m: - if close_original_fd: - f.close() - self.assertEqual(len(m), size) - with self.assertRaises(ValueError): - m.size() - with self.assertRaises(ValueError): - m.resize(size * 2) - with self.assertRaises(ValueError): - m.resize(size // 2) - self.assertEqual(m.closed, False) - - # Smoke-test other API - m.write_byte(ord('X')) - m[2] = ord('Y') - m.flush() - with open(TESTFN, "rb") as f: - self.assertEqual(f.read(4), b'XaYa') - self.assertEqual(m.tell(), 1) - m.seek(0) - self.assertEqual(m.tell(), 0) - self.assertEqual(m.read_byte(), ord('X')) - - self.assertEqual(m.closed, True) - self.assertEqual(os.stat(TESTFN).st_size, size) - - @unittest.skipIf(os.name == 'nt', 'trackfd not present on Windows') + with open(TESTFN, "r+b") as f: + with mmap.mmap(f.fileno(), size, trackfd=False) as m: + if close_original_fd: + f.close() + self.assertEqual(len(m), size) + with self.assertRaises(ValueError): + m.size() + with self.assertRaises(ValueError): + m.resize(size * 2) + with self.assertRaises(ValueError): + m.resize(size // 2) + self.assertIs(m.closed, False) + + # Smoke-test other API + m.write_byte(ord('X')) + m[2] = ord('Y') + m.flush() + with open(TESTFN, "rb") as f: + self.assertEqual(f.read(4), b'XaYa') + self.assertEqual(m.tell(), 1) + m.seek(0) + self.assertEqual(m.tell(), 0) + self.assertEqual(m.read_byte(), ord('X')) + + if os.name == 'nt' and not close_original_fd: + self.assertRaises(PermissionError, os.rename, TESTFN, TESTFN+'1') + else: + os.rename(TESTFN, TESTFN+'1') + os.rename(TESTFN+'1', TESTFN) + + self.assertIs(m.closed, True) + self.assertEqual(os.stat(TESTFN).st_size, size) + def test_trackfd_neg1(self): size = 64 with mmap.mmap(-1, size, trackfd=False) as m: @@ -315,15 +319,6 @@ def test_trackfd_neg1(self): m[0] = ord('a') assert m[0] == ord('a') - @unittest.skipIf(os.name != 'nt', 'trackfd only fails on Windows') - def test_no_trackfd_parameter_on_windows(self): - # 'trackffd' is an invalid keyword argument for this function - size = 64 - with self.assertRaises(TypeError): - mmap.mmap(-1, size, trackfd=True) - with self.assertRaises(TypeError): - mmap.mmap(-1, size, trackfd=False) - def test_bad_file_desc(self): # Try opening a bad file descriptor... self.assertRaises(OSError, mmap.mmap, -2, 4096) diff --git a/Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst b/Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst new file mode 100644 index 00000000000000..1043ee3310558f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2025-08-29-12-05-33.gh-issue-78502.VpIMxg.rst @@ -0,0 +1,2 @@ +:class:`mmap.mmap` now has a *trackfd* parameter on Windows; if it is +``False``, the file handle corresponding to *fileno* will not be duplicated. diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 1c300546c33fe8..dcaadb818e0bf7 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -119,12 +119,12 @@ typedef struct { #ifdef UNIX int fd; - _Bool trackfd; int flags; #endif PyObject *weakreflist; access_mode access; + _Bool trackfd; } mmap_object; #define mmap_object_CAST(op) ((mmap_object *)(op)) @@ -636,13 +636,11 @@ is_resizeable(mmap_object *self) "mmap can't resize with extant buffers exported."); return 0; } -#ifdef UNIX if (!self->trackfd) { PyErr_SetString(PyExc_ValueError, "mmap can't resize with trackfd=False."); return 0; } -#endif if ((self->access == ACCESS_WRITE) || (self->access == ACCESS_DEFAULT)) return 1; PyErr_Format(PyExc_TypeError, @@ -734,8 +732,6 @@ mmap_size_method(PyObject *op, PyObject *Py_UNUSED(ignored)) return PyLong_FromLong((long)low); size = (((long long)high)<<32) + low; return PyLong_FromLongLong(size); - } else { - return PyLong_FromSsize_t(self->size); } #endif /* MS_WINDOWS */ @@ -750,6 +746,7 @@ mmap_size_method(PyObject *op, PyObject *Py_UNUSED(ignored)) return PyLong_FromLong(status.st_size); #endif } +#endif /* UNIX */ else if (self->trackfd) { return PyLong_FromSsize_t(self->size); } @@ -758,7 +755,6 @@ mmap_size_method(PyObject *op, PyObject *Py_UNUSED(ignored)) "can't get size with trackfd=False"); return NULL; } -#endif /* UNIX */ } /* This assumes that you want the entire file mapped, @@ -1476,7 +1472,7 @@ static PyObject * new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict); PyDoc_STRVAR(mmap_doc, -"Windows: mmap(fileno, length[, tagname[, access[, offset]]])\n\ +"Windows: mmap(fileno, length[, tagname[, access[, offset[, trackfd]]]])\n\ \n\ Maps length bytes from the file specified by the file handle fileno,\n\ and returns a mmap object. If length is larger than the current size\n\ @@ -1737,16 +1733,17 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) PyObject *tagname = Py_None; DWORD dwErr = 0; int fileno; - HANDLE fh = 0; + HANDLE fh = INVALID_HANDLE_VALUE; int access = (access_mode)ACCESS_DEFAULT; + int trackfd = 1; DWORD flProtect, dwDesiredAccess; static char *keywords[] = { "fileno", "length", "tagname", - "access", "offset", NULL }; + "access", "offset", "trackfd", NULL }; - if (!PyArg_ParseTupleAndKeywords(args, kwdict, "in|OiL", keywords, + if (!PyArg_ParseTupleAndKeywords(args, kwdict, "in|OiL$p", keywords, &fileno, &map_size, - &tagname, &access, &offset)) { + &tagname, &access, &offset, &trackfd)) { return NULL; } @@ -1813,22 +1810,27 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) m_obj->map_handle = NULL; m_obj->tagname = NULL; m_obj->offset = offset; + m_obj->trackfd = trackfd; - if (fh) { - /* It is necessary to duplicate the handle, so the - Python code can close it on us */ - if (!DuplicateHandle( - GetCurrentProcess(), /* source process handle */ - fh, /* handle to be duplicated */ - GetCurrentProcess(), /* target proc handle */ - (LPHANDLE)&m_obj->file_handle, /* result */ - 0, /* access - ignored due to options value */ - FALSE, /* inherited by child processes? */ - DUPLICATE_SAME_ACCESS)) { /* options */ - dwErr = GetLastError(); - Py_DECREF(m_obj); - PyErr_SetFromWindowsErr(dwErr); - return NULL; + if (fh != INVALID_HANDLE_VALUE) { + if (trackfd) { + /* It is necessary to duplicate the handle, so the + Python code can close it on us */ + if (!DuplicateHandle( + GetCurrentProcess(), /* source process handle */ + fh, /* handle to be duplicated */ + GetCurrentProcess(), /* target proc handle */ + &fh, /* result */ + 0, /* access - ignored due to options value */ + FALSE, /* inherited by child processes? */ + DUPLICATE_SAME_ACCESS)) /* options */ + { + dwErr = GetLastError(); + Py_DECREF(m_obj); + PyErr_SetFromWindowsErr(dwErr); + return NULL; + } + m_obj->file_handle = fh; } if (!map_size) { DWORD low,high; @@ -1836,7 +1838,8 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) /* low might just happen to have the value INVALID_FILE_SIZE; so we need to check the last error also. */ if (low == INVALID_FILE_SIZE && - (dwErr = GetLastError()) != NO_ERROR) { + (dwErr = GetLastError()) != NO_ERROR) + { Py_DECREF(m_obj); return PyErr_SetFromWindowsErr(dwErr); } @@ -1898,7 +1901,7 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict) off_lo = (DWORD)(offset & 0xFFFFFFFF); /* For files, it would be sufficient to pass 0 as size. For anonymous maps, we have to pass the size explicitly. */ - m_obj->map_handle = CreateFileMappingW(m_obj->file_handle, + m_obj->map_handle = CreateFileMappingW(fh, NULL, flProtect, size_hi, From 974532e75888a82adbaa11c832fe67c132832b65 Mon Sep 17 00:00:00 2001 From: Cody Maloney Date: Wed, 3 Sep 2025 02:37:06 -0700 Subject: [PATCH 6/6] gh-138013: Move I/O tests to test_io (#138365) Centralize `io` tests into the `test_io` module so they are easier to find and work on. This will make it easier to split `test_general` which takes 30+ seconds in a debug build on my machine. This renames `test_bufio` to be `test_bufferedio` so that it matches the implementation file name (`bufferedio.c`). Validation performed: Tests are run in parallel after change: ```bash ./python.exe -m test test_io -uall,largefile,extralargefile -M12G -j8 ``` Docstring reformat in `test_io/__init__.py` looks reasonable: ```python >>> import test.test_io >>> help(test.test_io) ``` Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- Lib/test/libregrtest/findtests.py | 1 + Lib/test/test_io/__init__.py | 23 +++++++++++++++++ .../test_bufferedio.py} | 0 Lib/test/{ => test_io}/test_file.py | 0 Lib/test/{ => test_io}/test_fileio.py | 0 Lib/test/test_io/test_general.py | 25 +++---------------- Lib/test/{ => test_io}/test_largefile.py | 0 Lib/test/{ => test_io}/test_memoryio.py | 0 Lib/test/{ => test_io}/test_univnewlines.py | 0 9 files changed, 28 insertions(+), 21 deletions(-) rename Lib/test/{test_bufio.py => test_io/test_bufferedio.py} (100%) rename Lib/test/{ => test_io}/test_file.py (100%) rename Lib/test/{ => test_io}/test_fileio.py (100%) rename Lib/test/{ => test_io}/test_largefile.py (100%) rename Lib/test/{ => test_io}/test_memoryio.py (100%) rename Lib/test/{ => test_io}/test_univnewlines.py (100%) diff --git a/Lib/test/libregrtest/findtests.py b/Lib/test/libregrtest/findtests.py index f01c1240774707..79afaf9083ae59 100644 --- a/Lib/test/libregrtest/findtests.py +++ b/Lib/test/libregrtest/findtests.py @@ -24,6 +24,7 @@ "test_future_stmt", "test_gdb", "test_inspect", + "test_io", "test_pydoc", "test_multiprocessing_fork", "test_multiprocessing_forkserver", diff --git a/Lib/test/test_io/__init__.py b/Lib/test/test_io/__init__.py index 4b16ecc31156a5..c94fad3e779381 100644 --- a/Lib/test/test_io/__init__.py +++ b/Lib/test/test_io/__init__.py @@ -1,3 +1,26 @@ +"""Tests for the io module and its implementations (_io and _pyio) + +Tests are split across multiple files to increase +parallelism and focus on specific implementation pieces. + +* test_io + * test_bufferedio - tests file buffering + * test_memoryio - tests BytesIO and StringIO + * test_fileio - tests FileIO + * test_file - tests the file interface + * test_general - tests everything else in the io module + * test_univnewlines - tests universal newline support + * test_largefile - tests operations on a file greater than 2**32 bytes + (only enabled with -ulargefile) +* test_free_threading/test_io - tests thread safety of io objects + +.. attention:: + When writing tests for io, it's important to test both the C and Python + implementations. This is usually done by writing a base test that refers to + the type it is testing as an attribute. Then it provides custom subclasses to + test both implementations. This directory contains lots of examples. +""" + import os from test.support import load_package_tests diff --git a/Lib/test/test_bufio.py b/Lib/test/test_io/test_bufferedio.py similarity index 100% rename from Lib/test/test_bufio.py rename to Lib/test/test_io/test_bufferedio.py diff --git a/Lib/test/test_file.py b/Lib/test/test_io/test_file.py similarity index 100% rename from Lib/test/test_file.py rename to Lib/test/test_io/test_file.py diff --git a/Lib/test/test_fileio.py b/Lib/test/test_io/test_fileio.py similarity index 100% rename from Lib/test/test_fileio.py rename to Lib/test/test_io/test_fileio.py diff --git a/Lib/test/test_io/test_general.py b/Lib/test/test_io/test_general.py index e3d7d26a7e0f9c..30fe1e2f866091 100644 --- a/Lib/test/test_io/test_general.py +++ b/Lib/test/test_io/test_general.py @@ -1,24 +1,7 @@ -"""Unit tests for the io module.""" - -# Tests of io are scattered over the test suite: -# * test_bufio - tests file buffering -# * test_memoryio - tests BytesIO and StringIO -# * test_fileio - tests FileIO -# * test_file - tests the file interface -# * test_io.test_general - tests everything else in the io module -# * test_univnewlines - tests universal newline support -# * test_largefile - tests operations on a file greater than 2**32 bytes -# (only enabled with -ulargefile) -# * test_free_threading/test_io - tests thread safety of io objects - -################################################################################ -# ATTENTION TEST WRITERS!!! -################################################################################ -# When writing tests for io, it's important to test both the C and Python -# implementations. This is usually done by writing a base test that refers to -# the type it is testing as an attribute. Then it provides custom subclasses to -# test both implementations. This file has lots of examples. -################################################################################ +"""General tests for the io module. + +New tests should go in more specific modules; see test_io/__init__.py +""" import abc import array diff --git a/Lib/test/test_largefile.py b/Lib/test/test_io/test_largefile.py similarity index 100% rename from Lib/test/test_largefile.py rename to Lib/test/test_io/test_largefile.py diff --git a/Lib/test/test_memoryio.py b/Lib/test/test_io/test_memoryio.py similarity index 100% rename from Lib/test/test_memoryio.py rename to Lib/test/test_io/test_memoryio.py diff --git a/Lib/test/test_univnewlines.py b/Lib/test/test_io/test_univnewlines.py similarity index 100% rename from Lib/test/test_univnewlines.py rename to Lib/test/test_io/test_univnewlines.py