Merge upstream/main into file-output

Merged changes from upstream/main including:
- Subprocess enumeration functionality (get_child_pids, is_python_process)
- Various fixes and improvements

Combined with file-output branch features:
- Binary I/O writer and reader for profiling data
- Binary format export/replay support

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

Author: pablogsal

.gitignore

@@ -45,6 +45,7 @@ gmon.out
 .pytest_cache/
 .ruff_cache/
 .DS_Store
+.pixi/
 
 *.exe
 

Doc/deprecations/c-api-pending-removal-in-3.20.rst

@@ -3,7 +3,7 @@ Pending removal in Python 3.20
 
 * :c:func:`!_PyObject_CallMethodId`, :c:func:`!_PyObject_GetAttrId` and
   :c:func:`!_PyUnicode_FromId` are deprecated since 3.15 and will be removed in
-  3.20. Instead, use :c:func:`PyUnicode_FromString()` and cache the result in
+  3.20. Instead, use :c:func:`PyUnicode_InternFromString()` and cache the result in
   the module state, then call :c:func:`PyObject_CallMethod` or
   :c:func:`PyObject_GetAttr`.
   (Contributed by Victor Stinner in :gh:`141049`.)

Doc/deprecations/pending-removal-in-3.20.rst

@@ -7,6 +7,7 @@ Pending removal in Python 3.20
 
   - :mod:`argparse`
   - :mod:`csv`
+  - :mod:`ctypes`
   - :mod:`!ctypes.macholib`
   - :mod:`decimal` (use :data:`decimal.SPEC_VERSION` instead)
   - :mod:`http.server`

Doc/library/ctypes.rst

@@ -1388,6 +1388,9 @@ On Linux, :func:`~ctypes.util.find_library` tries to run external programs
 (``/sbin/ldconfig``, ``gcc``, ``objdump`` and ``ld``) to find the library file.
 It returns the filename of the library file.
 
+Note that if the output of these programs does not correspond to the dynamic
+linker used by Python, the result of this function may be misleading.
+
 .. versionchanged:: 3.6
    On Linux, the value of the environment variable ``LD_LIBRARY_PATH`` is used
    when searching for libraries, if a library cannot be found by any other means.
@@ -2132,6 +2135,8 @@ Utility functions
 
    The exact functionality is system dependent.
 
+   See :ref:`ctypes-finding-shared-libraries` for complete documentation.
+
 
 .. function:: find_msvcrt()
    :module: ctypes.util

Doc/library/profiling.sampling.rst

@@ -325,21 +325,23 @@ The default configuration works well for most use cases:
    :widths: 25 75
 
    * - Option
-     - Default behavior
-   * - ``--interval`` / ``-i``
+     - Default
+   * - Default for ``--interval`` / ``-i``
      - 100 µs between samples (~10,000 samples/sec)
-   * - ``--duration`` / ``-d``
-     - Profile for 10 seconds
-   * - ``--all-threads`` / ``-a``
-     - Sample main thread only
-   * - ``--native``
+   * - Default for ``--duration`` / ``-d``
+     - 10 seconds
+   * - Default for ``--all-threads`` / ``-a``
+     - Main thread only
+   * - Default for ``--native``
      - No ``<native>`` frames (C code time attributed to caller)
-   * - ``--no-gc``
-     - Include ``<GC>`` frames when garbage collection is active
-   * - ``--mode``
+   * - Default for ``--no-gc``
+     - ``<GC>`` frames included when garbage collection is active
+   * - Default for ``--mode``
      - Wall-clock mode (all samples recorded)
-   * - ``--realtime-stats``
-     - No live statistics display during profiling
+   * - Default for ``--realtime-stats``
+     - Disabled
+   * - Default for ``--subprocesses``
+     - Disabled
 
 
 Sampling interval and duration
@@ -472,6 +474,78 @@ working correctly and that sufficient samples are being collected. See
 :ref:`sampling-efficiency` for details on interpreting these metrics.
 
 
+Subprocess profiling
+--------------------
+
+The :option:`--subprocesses` option enables automatic profiling of subprocesses
+spawned by the target::
+
+   python -m profiling.sampling run --subprocesses script.py
+   python -m profiling.sampling attach --subprocesses 12345
+
+When enabled, the profiler monitors the target process for child process
+creation. When a new Python child process is detected, a separate profiler
+instance is automatically spawned to profile it. This is useful for
+applications that use :mod:`multiprocessing`, :mod:`subprocess`,
+:mod:`concurrent.futures` with :class:`~concurrent.futures.ProcessPoolExecutor`,
+or other process spawning mechanisms.
+
+.. code-block:: python
+   :caption: worker_pool.py
+
+   from concurrent.futures import ProcessPoolExecutor
+   import math
+
+   def compute_factorial(n):
+       total = 0
+       for i in range(50):
+           total += math.factorial(n)
+       return total
+
+   if __name__ == "__main__":
+       numbers = [5000 + i * 100 for i in range(50)]
+       with ProcessPoolExecutor(max_workers=4) as executor:
+           results = list(executor.map(compute_factorial, numbers))
+       print(f"Computed {len(results)} factorials")
+
+::
+
+   python -m profiling.sampling run --subprocesses --flamegraph worker_pool.py
+
+This produces separate flame graphs for the main process and each worker
+process: ``flamegraph_<main_pid>.html``, ``flamegraph_<worker1_pid>.html``,
+and so on.
+
+Each subprocess receives its own output file. The filename is derived from
+the specified output path (or the default) with the subprocess's process ID
+appended:
+
+- If you specify ``-o profile.html``, subprocesses produce ``profile_12345.html``,
+  ``profile_12346.html``, and so on
+- With default output, subprocesses produce files like ``flamegraph_12345.html``
+  or directories like ``heatmap_12345``
+- For pstats format (which defaults to stdout), subprocesses produce files like
+  ``profile_12345.pstats``
+
+The subprocess profilers inherit most sampling options from the parent (interval,
+duration, thread selection, native frames, GC frames, async-aware mode, and
+output format). All Python descendant processes are profiled recursively,
+including grandchildren and further descendants.
+
+Subprocess detection works by periodically scanning for new descendants of
+the target process and checking whether each new process is a Python process
+by probing the process memory for Python runtime structures. Non-Python
+subprocesses (such as shell commands or external tools) are ignored.
+
+There is a limit of 100 concurrent subprocess profilers to prevent resource
+exhaustion in programs that spawn many processes. If this limit is reached,
+additional subprocesses are not profiled and a warning is printed.
+
+The :option:`--subprocesses` option is incompatible with :option:`--live` mode
+because live mode uses an interactive terminal interface that cannot
+accommodate multiple concurrent profiler displays.
+
+
 .. _sampling-efficiency:
 
 Sampling efficiency
@@ -1302,6 +1376,11 @@ Sampling options
    Compatible with ``--live``, ``--flamegraph``, ``--heatmap``, and ``--gecko``
    formats only.
 
+.. option:: --subprocesses
+
+   Also profile subprocesses. Each subprocess gets its own profiler
+   instance and output file. Incompatible with ``--live``.
+
 
 Mode options
 ------------

Doc/whatsnew/3.15.rst

@@ -74,6 +74,8 @@ Summary -- Release highlights
 * :pep:`782`: :ref:`A new PyBytesWriter C API to create a Python bytes object
   <whatsnew315-pep782>`
 * :ref:`Improved error messages <whatsnew315-improved-error-messages>`
+* :ref:`__pycache__ directories now contain a .gitignore file
+  <whatsnew315-pycache-gitignore>`
 
 
 New features
@@ -397,6 +399,12 @@ Other language changes
   for any class.
   (Contributed by Serhiy Storchaka in :gh:`41779`.)
 
+.. _whatsnew315-pycache-gitignore:
+
+* :file:`__pycache__` directories now contain a :file:`.gitignore` file for Git
+  that ignores their contents.
+  (Contributed by Stan Ulbrych in :gh:`141081`.)
+
 
 New modules
 ===========
@@ -1024,6 +1032,7 @@ New deprecations
 
     - :mod:`argparse`
     - :mod:`csv`
+    - :mod:`ctypes`
     - :mod:`!ctypes.macholib`
     - :mod:`decimal` (use :data:`decimal.SPEC_VERSION` instead)
     - :mod:`http.server`
@@ -1217,7 +1226,7 @@ Deprecated C APIs
 
 * :c:func:`!_PyObject_CallMethodId`, :c:func:`!_PyObject_GetAttrId` and
   :c:func:`!_PyUnicode_FromId` are deprecated since 3.15 and will be removed in
-  3.20. Instead, use :c:func:`PyUnicode_FromString()` and cache the result in
+  3.20. Instead, use :c:func:`PyUnicode_InternFromString()` and cache the result in
   the module state, then call :c:func:`PyObject_CallMethod` or
   :c:func:`PyObject_GetAttr`.
   (Contributed by Victor Stinner in :gh:`141049`.)

Include/internal/pycore_context.h

@@ -55,5 +55,8 @@ struct _pycontexttokenobject {
 // Export for '_testcapi' shared extension
 PyAPI_FUNC(PyObject*) _PyContext_NewHamtForTests(void);
 
+PyAPI_FUNC(int) _PyContext_Enter(PyThreadState *ts, PyObject *octx);
+PyAPI_FUNC(int) _PyContext_Exit(PyThreadState *ts, PyObject *octx);
+
 
 #endif /* !Py_INTERNAL_CONTEXT_H */

Include/internal/pycore_global_objects_fini_generated.h

@@ -722,6 +722,7 @@ struct _Py_global_strings {
         STRUCT_FOR_ID(readline)
         STRUCT_FOR_ID(readonly)
         STRUCT_FOR_ID(real)
+        STRUCT_FOR_ID(recursive)
         STRUCT_FOR_ID(reducer_override)
         STRUCT_FOR_ID(registry)
         STRUCT_FOR_ID(rel_tol)

Include/internal/pycore_global_strings.h

@@ -496,6 +496,9 @@ static inline void Py_DECREF_MORTAL_SPECIALIZED(PyObject *op, destructor destruc
 #define Py_DECREF_MORTAL_SPECIALIZED(op, destruct) Py_DECREF_MORTAL_SPECIALIZED(_PyObject_CAST(op), destruct)
 
 #endif
+#else  // Py_GIL_DISABLED
+# define Py_DECREF_MORTAL(op) Py_DECREF(op)
+# define Py_DECREF_MORTAL_SPECIALIZED(op, destruct) Py_DECREF(op)
 #endif
 
 /* Inline functions trading binary compatibility for speed:
@@ -1045,6 +1048,8 @@ static inline Py_ALWAYS_INLINE void _Py_INCREF_MORTAL(PyObject *op)
     }
 #endif
 }
+#else
+# define _Py_INCREF_MORTAL(op) Py_INCREF(op)
 #endif
 
 /* Utility for the tp_traverse slot of mutable heap types that have no other