[pull] main from python:main

Doc/library/io.rst

@@ -528,14 +528,13 @@ I/O Base Classes
    It inherits from :class:`IOBase`.
 
    The main difference with :class:`RawIOBase` is that methods :meth:`read`,
-   :meth:`readinto` and :meth:`write` will try (respectively) to read as much
-   input as requested or to consume all given output, at the expense of
-   making perhaps more than one system call.
+   :meth:`readinto` and :meth:`write` will try (respectively) to read
+   as much input as requested or to emit all provided data.
 
-   In addition, those methods can raise :exc:`BlockingIOError` if the
-   underlying raw stream is in non-blocking mode and cannot take or give
-   enough data; unlike their :class:`RawIOBase` counterparts, they will
-   never return ``None``.
+   In addition, if the underlying raw stream is in non-blocking mode, when the
+   system returns would block :meth:`write` will raise :exc:`BlockingIOError`
+   with :attr:`BlockingIOError.characters_written` and :meth:`read` will return
+   data read so far or ``None`` if no data is available.
 
    Besides, the :meth:`read` method does not have a default
    implementation that defers to :meth:`readinto`.
@@ -568,29 +567,40 @@ I/O Base Classes
 
    .. method:: read(size=-1, /)
 
-      Read and return up to *size* bytes.  If the argument is omitted, ``None``,
-      or negative, data is read and returned until EOF is reached.  An empty
-      :class:`bytes` object is returned if the stream is already at EOF.
+      Read and return up to *size* bytes. If the argument is omitted, ``None``,
+      or negative read as much as possible.
 
-      If the argument is positive, and the underlying raw stream is not
-      interactive, multiple raw reads may be issued to satisfy the byte count
-      (unless EOF is reached first).  But for interactive raw streams, at most
-      one raw read will be issued, and a short result does not imply that EOF is
-      imminent.
+      Fewer bytes may be returned than requested. An empty :class:`bytes` object
+      is returned if the stream is already at EOF. More than one read may be
+      made and calls may be retried if specific errors are encountered, see
+      :meth:`os.read` and :pep:`475` for more details. Less than size bytes
+      being returned does not imply that EOF is imminent.
 
-      A :exc:`BlockingIOError` is raised if the underlying raw stream is in
-      non blocking-mode, and has no data available at the moment.
+      When reading as much as possible the default implementation will use
+      ``raw.readall`` if available (which should implement
+      :meth:`RawIOBase.readall`), otherwise will read in a loop until read
+      returns ``None``, an empty :class:`bytes`, or a non-retryable error. For
+      most streams this is to EOF, but for non-blocking streams more data may
+      become available.
+
+      .. note::
+
+         When the underlying raw stream is non-blocking, implementations may
+         either raise :exc:`BlockingIOError` or return ``None`` if no data is
+         available. :mod:`io` implementations return ``None``.
 
    .. method:: read1(size=-1, /)
 
-      Read and return up to *size* bytes, with at most one call to the
-      underlying raw stream's :meth:`~RawIOBase.read` (or
-      :meth:`~RawIOBase.readinto`) method.  This can be useful if you are
-      implementing your own buffering on top of a :class:`BufferedIOBase`
-      object.
+      Read and return up to *size* bytes, calling :meth:`~RawIOBase.readinto`
+      which may retry if :py:const:`~errno.EINTR` is encountered per
+      :pep:`475`. If *size* is ``-1`` or not provided, the implementation will
+      choose an arbitrary value for *size*.
 
-      If *size* is ``-1`` (the default), an arbitrary number of bytes are
-      returned (more than zero unless EOF is reached).
+      .. note::
+
+         When the underlying raw stream is non-blocking, implementations may
+         either raise :exc:`BlockingIOError` or return ``None`` if no data is
+         available. :mod:`io` implementations return ``None``.
 
    .. method:: readinto(b, /)
 
@@ -767,34 +777,21 @@ than raw I/O does.
 
    .. method:: peek(size=0, /)
 
-      Return bytes from the stream without advancing the position.  At most one
-      single read on the raw stream is done to satisfy the call. The number of
-      bytes returned may be less or more than requested.
+      Return bytes from the stream without advancing the position. The number of
+      bytes returned may be less or more than requested. If the underlying raw
+      stream is non-blocking and the operation would block, returns empty bytes.
 
    .. method:: read(size=-1, /)
 
-      Read and return *size* bytes, or if *size* is not given or negative, until
-      EOF or if the read call would block in non-blocking mode.
-
-      .. note::
-
-         When the underlying raw stream is non-blocking, a :exc:`BlockingIOError`
-         may be raised if a read operation cannot be completed immediately.
+      In :class:`BufferedReader` this is the same as :meth:`io.BufferedIOBase.read`
 
    .. method:: read1(size=-1, /)
 
-      Read and return up to *size* bytes with only one call on the raw stream.
-      If at least one byte is buffered, only buffered bytes are returned.
-      Otherwise, one raw stream read call is made.
+      In :class:`BufferedReader` this is the same as :meth:`io.BufferedIOBase.read1`
 
       .. versionchanged:: 3.7
          The *size* argument is now optional.
 
-      .. note::
-
-         When the underlying raw stream is non-blocking, a :exc:`BlockingIOError`
-         may be raised if a read operation cannot be completed immediately.
-
 .. class:: BufferedWriter(raw, buffer_size=DEFAULT_BUFFER_SIZE)
 
    A buffered binary stream providing higher-level access to a writeable, non
@@ -826,8 +823,8 @@ than raw I/O does.
 
       Write the :term:`bytes-like object`, *b*, and return the
       number of bytes written.  When in non-blocking mode, a
-      :exc:`BlockingIOError` is raised if the buffer needs to be written out but
-      the raw stream blocks.
+      :exc:`BlockingIOError` with :attr:`BlockingIOError.characters_written` set
+      is raised if the buffer needs to be written out but the raw stream blocks.
 
 
 .. class:: BufferedRandom(raw, buffer_size=DEFAULT_BUFFER_SIZE)

Include/internal/pycore_pyerrors.h

@@ -94,13 +94,13 @@ extern void _PyErr_Fetch(
     PyObject **value,
     PyObject **traceback);
 
-extern PyObject* _PyErr_GetRaisedException(PyThreadState *tstate);
+PyAPI_FUNC(PyObject*) _PyErr_GetRaisedException(PyThreadState *tstate);
 
 PyAPI_FUNC(int) _PyErr_ExceptionMatches(
     PyThreadState *tstate,
     PyObject *exc);
 
-extern void _PyErr_SetRaisedException(PyThreadState *tstate, PyObject *exc);
+PyAPI_FUNC(void) _PyErr_SetRaisedException(PyThreadState *tstate, PyObject *exc);
 
 extern void _PyErr_Restore(
     PyThreadState *tstate,

Lib/multiprocessing/connection.py

@@ -76,7 +76,7 @@ def arbitrary_address(family):
     if family == 'AF_INET':
         return ('localhost', 0)
     elif family == 'AF_UNIX':
-        return tempfile.mktemp(prefix='listener-', dir=util.get_temp_dir())
+        return tempfile.mktemp(prefix='sock-', dir=util.get_temp_dir())
     elif family == 'AF_PIPE':
         return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
                                (os.getpid(), next(_mmap_counter)), dir="")

Lib/multiprocessing/util.py

@@ -19,7 +19,7 @@
 from . import process
 
 __all__ = [
-    'sub_debug', 'debug', 'info', 'sub_warning', 'get_logger',
+    'sub_debug', 'debug', 'info', 'sub_warning', 'warn', 'get_logger',
     'log_to_stderr', 'get_temp_dir', 'register_after_fork',
     'is_exiting', 'Finalize', 'ForkAwareThreadLock', 'ForkAwareLocal',
     'close_all_fds_except', 'SUBDEBUG', 'SUBWARNING',
@@ -34,6 +34,7 @@
 DEBUG = 10
 INFO = 20
 SUBWARNING = 25
+WARNING = 30
 
 LOGGER_NAME = 'multiprocessing'
 DEFAULT_LOGGING_FORMAT = '[%(levelname)s/%(processName)s] %(message)s'
@@ -53,6 +54,10 @@ def info(msg, *args):
     if _logger:
         _logger.log(INFO, msg, *args, stacklevel=2)
 
+def warn(msg, *args):
+    if _logger:
+        _logger.log(WARNING, msg, *args, stacklevel=2)
+
 def sub_warning(msg, *args):
     if _logger:
         _logger.log(SUBWARNING, msg, *args, stacklevel=2)
@@ -121,6 +126,21 @@ def is_abstract_socket_namespace(address):
 # Function returning a temp directory which will be removed on exit
 #
 
+# Maximum length of a socket file path is usually between 92 and 108 [1],
+# but Linux is known to use a size of 108 [2]. BSD-based systems usually
+# use a size of 104 or 108 and Windows does not create AF_UNIX sockets.
+#
+# [1]: https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sys_un.h.html
+# [2]: https://man7.org/linux/man-pages/man7/unix.7.html.
+
+if sys.platform == 'linux':
+    _SUN_PATH_MAX = 108
+elif sys.platform.startswith(('openbsd', 'freebsd')):
+    _SUN_PATH_MAX = 104
+else:
+    # On Windows platforms, we do not create AF_UNIX sockets.
+    _SUN_PATH_MAX = None if os.name == 'nt' else 92
+
 def _remove_temp_dir(rmtree, tempdir):
     rmtree(tempdir)
 
@@ -130,12 +150,67 @@ def _remove_temp_dir(rmtree, tempdir):
     if current_process is not None:
         current_process._config['tempdir'] = None
 
+def _get_base_temp_dir(tempfile):
+    """Get a temporary directory where socket files will be created.
+
+    To prevent additional imports, pass a pre-imported 'tempfile' module.
+    """
+    if os.name == 'nt':
+        return None
+    # Most of the time, the default temporary directory is /tmp. Thus,
+    # listener sockets files "$TMPDIR/pymp-XXXXXXXX/sock-XXXXXXXX" do
+    # not have a path length exceeding SUN_PATH_MAX.
+    #
+    # If users specify their own temporary directory, we may be unable
+    # to create those files. Therefore, we fall back to the system-wide
+    # temporary directory /tmp, assumed to exist on POSIX systems.
+    #
+    # See https://github.com/python/cpython/issues/132124.
+    base_tempdir = tempfile.gettempdir()
+    # Files created in a temporary directory are suffixed by a string
+    # generated by tempfile._RandomNameSequence, which, by design,
+    # is 8 characters long.
+    #
+    # Thus, the length of socket filename will be:
+    #
+    #   len(base_tempdir + '/pymp-XXXXXXXX' + '/sock-XXXXXXXX')
+    sun_path_len = len(base_tempdir) + 14 + 14
+    if sun_path_len <= _SUN_PATH_MAX:
+        return base_tempdir
+    # Fallback to the default system-wide temporary directory.
+    # This ignores user-defined environment variables.
+    #
+    # On POSIX systems, /tmp MUST be writable by any application [1].
+    # We however emit a warning if this is not the case to prevent
+    # obscure errors later in the execution.
+    #
+    # On some legacy systems, /var/tmp and /usr/tmp can be present
+    # and will be used instead.
+    #
+    # [1]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s18.html
+    dirlist = ['/tmp', '/var/tmp', '/usr/tmp']
+    try:
+        base_system_tempdir = tempfile._get_default_tempdir(dirlist)
+    except FileNotFoundError:
+        warn("Process-wide temporary directory %s will not be usable for "
+             "creating socket files and no usable system-wide temporary "
+             "directory was found in %s", base_tempdir, dirlist)
+        # At this point, the system-wide temporary directory is not usable
+        # but we may assume that the user-defined one is, even if we will
+        # not be able to write socket files out there.
+        return base_tempdir
+    warn("Ignoring user-defined temporary directory: %s", base_tempdir)
+    # at most max(map(len, dirlist)) + 14 + 14 = 36 characters
+    assert len(base_system_tempdir) + 14 + 14 <= _SUN_PATH_MAX
+    return base_system_tempdir
+
 def get_temp_dir():
     # get name of a temp directory which will be automatically cleaned up
     tempdir = process.current_process()._config.get('tempdir')
     if tempdir is None:
         import shutil, tempfile
-        tempdir = tempfile.mkdtemp(prefix='pymp-')
+        base_tempdir = _get_base_temp_dir(tempfile)
+        tempdir = tempfile.mkdtemp(prefix='pymp-', dir=base_tempdir)
         info('created temp directory %s', tempdir)
         # keep a strong reference to shutil.rmtree(), since the finalizer
         # can be called late during Python shutdown

Lib/tempfile.py

@@ -180,7 +180,7 @@ def _candidate_tempdir_list():
 
     return dirlist
 
-def _get_default_tempdir():
+def _get_default_tempdir(dirlist=None):
     """Calculate the default directory to use for temporary files.
     This routine should be called exactly once.
 
@@ -190,7 +190,8 @@ def _get_default_tempdir():
     service, the name of the test file must be randomized."""
 
     namer = _RandomNameSequence()
-    dirlist = _candidate_tempdir_list()
+    if dirlist is None:
+        dirlist = _candidate_tempdir_list()
 
     for dir in dirlist:
         if dir != _os.curdir:

Lib/test/test__interpreters.py

@@ -1054,7 +1054,7 @@ def test_closure(self):
         def script():
             assert spam
 
-        with self.assertRaises(ValueError):
+        with self.assertRaises(TypeError):
             _interpreters.run_func(self.id, script)
 
     # XXX This hasn't been fixed yet.
@@ -1065,6 +1065,7 @@ def script():
         with self.assertRaises(ValueError):
             _interpreters.run_func(self.id, script)
 
+    @unittest.skip("we're not quite there yet")
     def test_args(self):
         with self.subTest('args'):
             def script(a, b=0):

Misc/NEWS.d/next/Library/2025-05-16-12-40-37.gh-issue-132124.T_5Odx.rst

@@ -0,0 +1,6 @@
+On POSIX-compliant systems, :func:`!multiprocessing.util.get_temp_dir` now
+ignores :envvar:`TMPDIR` (and similar environment variables) if the path
+length of ``AF_UNIX`` socket files exceeds the platform-specific maximum
+length when using the :ref:`forkserver
+<multiprocessing-start-method-forkserver>` start method. Patch by Bénédikt
+Tran.