Add what's new

Author: pablogsal

Doc/whatsnew/3.15.rst

@@ -70,6 +70,99 @@ Summary --- release highlights
 New features
 ============
 
+.. _whatsnew314-sampling-profiler:
+
+Statistical sampling profiler for production debugging
+------------------------------------------------------
+
+A new statistical sampling profiler has been added to the :mod:`profile` module as 
+:mod:`profile.sample`. This profiler enables low-overhead performance analysis of 
+running Python processes without requiring code modification or process restart.
+
+Unlike deterministic profilers (:mod:`cProfile` and :mod:`profile`) that instrument 
+every function call, the sampling profiler periodically captures stack traces from 
+running processes.  This approach provides virtually zero overhead while achieving 
+sampling rates of **up to 200,000 Hz**, making it the fastest sampling profiler 
+available for Python (at the time of its contribution) and ideal for debugging
+performance issues in production environments.
+
+Key features include:
+
+* **Zero-overhead profiling**: Attach to any running Python process without 
+  affecting its performance
+* **No code modification required**: Profile existing applications without restart
+* **Real-time statistics**: Monitor sampling quality during data collection
+* **Multiple output formats**: Generate both detailed statistics and flamegraph data
+* **Thread-aware profiling**: Option to profile all threads or just the main thread
+
+Profile an existing process for 10 seconds::
+
+   python -m profile.sample 1234
+
+Profile with custom settings and real-time statistics::
+
+   python -m profile.sample --realtime-stats -i 50 -d 30 1234
+
+Generate flamegraph data::
+
+   python -m profile.sample --collapsed -o stacks.txt 1234
+
+The profiler generates statistical estimates of where time is spent::
+
+  Real-time sampling stats: Mean: 100261.5Hz (9.97µs) Min: 86333.4Hz (11.58µs) Max: 118807.2Hz (8.42µs) Samples: 400001
+  Captured 498841 samples in 5.00 seconds
+  Sample rate: 99768.04 samples/sec
+  Error rate: 0.72%
+  Profile Stats:
+        nsamples   sample%   tottime (s)    cumul%   cumtime (s)  filename:lineno(function)
+        43/418858       0.0         0.000      87.9         4.189  case.py:667(TestCase.run)
+      3293/418812       0.7         0.033      87.9         4.188  case.py:613(TestCase._callTestMethod)
+    158562/158562      33.3         1.586      33.3         1.586  test_compile.py:725(TestSpecifics.test_compiler_recursion_limit.<locals>.check_limit)
+    129553/129553      27.2         1.296      27.2         1.296  ast.py:46(parse)
+        0/128129       0.0         0.000      26.9         1.281  test_ast.py:884(AST_Tests.test_ast_recursion_limit.<locals>.check_limit)
+          7/67446       0.0         0.000      14.2         0.674  test_compile.py:729(TestSpecifics.test_compiler_recursion_limit)
+          6/60380       0.0         0.000      12.7         0.604  test_ast.py:888(AST_Tests.test_ast_recursion_limit)
+          3/50020       0.0         0.000      10.5         0.500  test_compile.py:727(TestSpecifics.test_compiler_recursion_limit)
+          1/38011       0.0         0.000       8.0         0.380  test_ast.py:886(AST_Tests.test_ast_recursion_limit)
+          1/25076       0.0         0.000       5.3         0.251  test_compile.py:728(TestSpecifics.test_compiler_recursion_limit)
+      22361/22362       4.7         0.224       4.7         0.224  test_compile.py:1368(TestSpecifics.test_big_dict_literal)
+          4/18008       0.0         0.000       3.8         0.180  test_ast.py:889(AST_Tests.test_ast_recursion_limit)
+        11/17696       0.0         0.000       3.7         0.177  subprocess.py:1038(Popen.__init__)
+      16968/16968       3.6         0.170       3.6         0.170  subprocess.py:1900(Popen._execute_child)
+          2/16941       0.0         0.000       3.6         0.169  test_compile.py:730(TestSpecifics.test_compiler_recursion_limit)
+
+  Legend:
+    nsamples: Direct/Cumulative samples (direct executing / on call stack)
+    sample%: Percentage of total samples this function was directly executing
+    tottime: Estimated total time spent directly in this function
+    cumul%: Percentage of total samples when this function was on the call stack
+    cumtime: Estimated cumulative time (including time in called functions)
+    filename:lineno(function): Function location and name
+
+  Summary of Interesting Functions:
+
+  Functions with Highest Direct/Cumulative Ratio (Hot Spots):
+    1.000 direct/cumulative ratio, 33.3% direct samples: test_compile.py:(TestSpecifics.test_compiler_recursion_limit.<locals>.check_limit)
+    1.000 direct/cumulative ratio, 27.2% direct samples: ast.py:(parse)
+    1.000 direct/cumulative ratio, 3.6% direct samples: subprocess.py:(Popen._execute_child)
+
+  Functions with Highest Call Frequency (Indirect Calls):
+    418815 indirect calls, 87.9% total stack presence: case.py:(TestCase.run)
+    415519 indirect calls, 87.9% total stack presence: case.py:(TestCase._callTestMethod)
+    159470 indirect calls, 33.5% total stack presence: test_compile.py:(TestSpecifics.test_compiler_recursion_limit)
+
+  Functions with Highest Call Magnification (Cumulative/Direct):
+    12267.9x call magnification, 159470 indirect calls from 13 direct: test_compile.py:(TestSpecifics.test_compiler_recursion_limit)
+    10581.7x call magnification, 116388 indirect calls from 11 direct: test_ast.py:(AST_Tests.test_ast_recursion_limit)
+    9740.9x call magnification, 418815 indirect calls from 43 direct: case.py:(TestCase.run)
+
+The profiler automatically identifies performance bottlenecks through statistical 
+analysis, highlighting functions with high CPU usage and call frequency patterns.
+
+This capability is particularly valuable for debugging performance issues in 
+production systems where traditional profiling approaches would be too intrusive.
+
+(Contributed by Pablo Galindo and László Kiss Kollár in :gh:`135953`.)
 
 
 Other language changes