Add docs

Author: pablogsal

Doc/library/profile.rst

@@ -4,7 +4,7 @@
 The Python Profilers
 ********************
 
-**Source code:** :source:`Lib/profile.py` and :source:`Lib/pstats.py`
+**Source code:** :source:`Lib/profile.py`, :source:`Lib/pstats.py`, and :source:`Lib/profile/sample.py`
 
 --------------
 
@@ -14,23 +14,59 @@ Introduction to the profilers
 =============================
 
 .. index::
+   single: statistical profiling
+   single: profiling, statistical
    single: deterministic profiling
    single: profiling, deterministic
 
-:mod:`cProfile` and :mod:`profile` provide :dfn:`deterministic profiling` of
+Python provides both :dfn:`statistical profiling` and :dfn:`deterministic profiling` of
 Python programs. A :dfn:`profile` is a set of statistics that describes how
 often and for how long various parts of the program executed. These statistics
 can be formatted into reports via the :mod:`pstats` module.
 
-The Python standard library provides two different implementations of the same
-profiling interface:
+**Profiler Comparison:**
+
++-------------------+----------------------+----------------------+----------------------+
+| Feature           | Statistical          | Deterministic        | Deterministic        |
+|                   | (``profile.sample``) | (``cProfile``)       | (``profile``)        |
++===================+======================+======================+======================+
+| **Target**        | Running process      | Code you run         | Code you run         |
++-------------------+----------------------+----------------------+----------------------+
+| **Overhead**      | Virtually none       | Moderate             | High                 |
++-------------------+----------------------+----------------------+----------------------+
+| **Accuracy**      | Statistical approx.  | Exact call counts    | Exact call counts    |
++-------------------+----------------------+----------------------+----------------------+
+| **Setup**         | Attach to any PID    | Instrument code      | Instrument code      |
++-------------------+----------------------+----------------------+----------------------+
+| **Use Case**      | Production debugging | Development/testing  | Profiler extension   |
++-------------------+----------------------+----------------------+----------------------+
+| **Implementation**| C extension          | C extension          | Pure Python          |
++-------------------+----------------------+----------------------+----------------------+
 
-1. :mod:`cProfile` is recommended for most users; it's a C extension with
+.. note::
+
+   The statistical profiler (:mod:`profile.sample`) is recommended for most production
+   use cases due to its extremely low overhead and ability to profile running processes
+   without modification. It can attach to any Python process and collect performance 
+   data with minimal impact on execution speed, making it ideal for debugging 
+   performance issues in live applications.
+
+The Python standard library provides three different profiling implementations:
+
+**Statistical Profiler:**
+
+1. :mod:`profile.sample` provides statistical profiling of running Python processes
+   using periodic stack sampling. It can attach to any running Python process without
+   requiring code modification or restart, making it ideal for production debugging.
+
+**Deterministic Profilers:**
+
+2. :mod:`cProfile` is recommended for development and testing; it's a C extension with
    reasonable overhead that makes it suitable for profiling long-running
    programs.  Based on :mod:`lsprof`, contributed by Brett Rosen and Ted
    Czotter.
 
-2. :mod:`profile`, a pure Python module whose interface is imitated by
+3. :mod:`profile`, a pure Python module whose interface is imitated by
    :mod:`cProfile`, but which adds significant overhead to profiled programs.
    If you're trying to extend the profiler in some way, the task might be easier
    with this module.  Originally designed and written by Jim Roskind.
@@ -44,6 +80,22 @@ profiling interface:
    but not for C-level functions, and so the C code would seem faster than any
    Python one.
 
+.. _statistical-profiling:
+
+What Is Statistical Profiling?
+==============================
+
+:dfn:`Statistical profiling` works by periodically interrupting a running program to capture its current call stack. Rather than monitoring every function entry and exit like deterministic profilers, it takes snapshots at regular intervals to build a statistical picture of where the program spends its time.
+
+The sampling profiler uses process memory reading (via system calls like `process_vm_readv` on Linux, `vm_read` on macOS, and `ReadProcessMemory` on Windows) to attach to a running Python process and extract stack trace information without requiring any code modification or restart of the target process. This approach provides several key advantages over traditional profiling methods.
+
+The fundamental principle is that if a function appears frequently in the collected stack samples, it is likely consuming significant CPU time. By analyzing thousands of samples, the profiler can accurately estimate the relative time spent in different parts of the program. The statistical nature means that while individual measurements may vary, the aggregate results converge to represent the true performance characteristics of the application.
+
+Since statistical profiling operates externally to the target process, it introduces virtually no overhead to the running program. The profiler process runs separately and reads the target process memory without interrupting its execution. This makes it suitable for profiling production systems where performance impact must be minimized.
+
+The accuracy of statistical profiling improves with the number of samples collected. Short-lived functions may be missed or underrepresented, while long-running functions will be captured proportionally to their execution time. This characteristic makes statistical profiling particularly effective for identifying the most significant performance bottlenecks rather than providing exhaustive coverage of all function calls.
+
+Statistical profiling excels at answering questions like "which functions consume the most CPU time?" and "where should I focus optimization efforts?" rather than "exactly how many times was this function called?" The trade-off between precision and practicality makes it an invaluable tool for performance analysis in real-world applications.
 
 .. _profile-instant:
 
@@ -54,6 +106,18 @@ This section is provided for users that "don't want to read the manual." It
 provides a very brief overview, and allows a user to rapidly perform profiling
 on an existing application.
 
+**Statistical Profiling (Recommended for Production):**
+
+To profile an existing running process::
+
+   python -m profile.sample 1234
+
+To profile with custom settings::
+
+   python -m profile.sample -i 50 -d 30 1234
+
+**Deterministic Profiling (Development/Testing):**
+
 To profile a function that takes a single argument, you can do::
 
    import cProfile
@@ -121,8 +185,134 @@ results to a file by specifying a filename to the :func:`run` function::
 The :class:`pstats.Stats` class reads profile results from a file and formats
 them in various ways.
 
+.. _sampling-profiler-cli:
+
+Statistical Profiler Command Line Interface
+===========================================
+
+.. program:: profile.sample
+
+The :mod:`profile.sample` module can be invoked as a script to profile running processes::
+
+   python -m profile.sample [options] PID
+
+**Basic Usage Examples:**
+
+Profile process 1234 for 10 seconds with default settings::
+
+   python -m profile.sample 1234
+
+Profile with custom interval and duration, save to file::
+
+   python -m profile.sample -i 50 -d 30 -o profile.stats 1234
+
+Generate collapsed stacks for flamegraph::
+
+   python -m profile.sample --collapsed 1234
+
+Profile all threads, sort by total time::
+
+   python -m profile.sample -a --sort-tottime 1234
+
+Profile with real-time sampling statistics::
+
+   python -m profile.sample --realtime-stats 1234
+
+**Command Line Options:**
+
+.. option:: PID
+
+   Process ID of the Python process to profile (required)
+
+.. option:: -i, --interval INTERVAL
+
+   Sampling interval in microseconds (default: 100)
+
+.. option:: -d, --duration DURATION
+
+   Sampling duration in seconds (default: 10)
+
+.. option:: -a, --all-threads
+
+   Sample all threads in the process instead of just the main thread
+
+.. option:: --realtime-stats
+
+   Print real-time sampling statistics during profiling
+
+.. option:: --pstats
+
+   Generate pstats output (default)
+
+.. option:: --collapsed
+
+   Generate collapsed stack traces for flamegraphs
+
+.. option:: -o, --outfile OUTFILE
+
+   Save output to a file
+
+**Sorting Options (pstats format only):**
+
+.. option:: --sort-nsamples
+
+   Sort by number of direct samples
+
+.. option:: --sort-tottime
+
+   Sort by total time
+
+.. option:: --sort-cumtime
+
+   Sort by cumulative time (default)
+
+.. option:: --sort-sample-pct
+
+   Sort by sample percentage
+
+.. option:: --sort-cumul-pct
+
+   Sort by cumulative sample percentage
+
+.. option:: --sort-nsamples-cumul
+
+   Sort by cumulative samples
+
+.. option:: --sort-name
+
+   Sort by function name
+
+.. option:: -l, --limit LIMIT
+
+   Limit the number of rows in the output (default: 15)
+
+.. option:: --no-summary
+
+   Disable the summary section in the output
+
+**Understanding Statistical Profile Output:**
+
+The statistical profiler produces output similar to deterministic profilers but with different column meanings::
+
+   Profile Stats:
+          nsamples  sample%     tottime (ms)  cumul%    cumtime (ms)  filename:lineno(function)
+             45/67     12.5        23.450     18.6        56.780     mymodule.py:42(process_data)
+             23/23      6.4        15.230      6.4        15.230     <built-in>:0(len)
+
+**Column Meanings:**
+
+- **nsamples**: `direct/cumulative` - Times function was directly executing / on call stack
+- **sample%**: Percentage of total samples where function was directly executing  
+- **tottime**: Estimated time spent directly in this function
+- **cumul%**: Percentage of samples where function was anywhere on call stack
+- **cumtime**: Estimated cumulative time including called functions
+- **filename:lineno(function)**: Location and name of the function
+
 .. _profile-cli:
 
+Deterministic Profiler Command Line Interface
+=============================================
+
 .. program:: cProfile
 
 The files :mod:`cProfile` and :mod:`profile` can also be invoked as a script to
@@ -564,7 +754,7 @@ What Is Deterministic Profiling?
 call*, *function return*, and *exception* events are monitored, and precise
 timings are made for the intervals between these events (during which time the
 user's code is executing).  In contrast, :dfn:`statistical profiling` (which is
-not done by this module) randomly samples the effective instruction pointer, and
+provided by the :mod:`profile.sample` module) periodically samples the effective instruction pointer, and
 deduces where time is being spent.  The latter technique traditionally involves
 less overhead (as the code does not need to be instrumented), but provides only
 relative indications of where time is being spent.
@@ -712,4 +902,4 @@ you are using :class:`profile.Profile` or :class:`cProfile.Profile`,
 
 Python 3.3 adds several new functions in :mod:`time` that can be used to make
 precise measurements of process or wall-clock time. For example, see
-:func:`time.perf_counter`.
+:func:`time.perf_counter`.