|
| 1 | +# Python sys.monitoring Tracer Test Design |
| 2 | + |
| 3 | +## Overview |
| 4 | +This document outlines a test suite for validating the Python tracer built on `sys.monitoring` and `runtime_tracing`. Each test item corresponds to roughly 1–10 lines of implementation and exercises tracer behavior under typical and edge conditions. |
| 5 | + |
| 6 | +## Setup |
| 7 | +- Establish a temporary directory for trace output and source snapshots. |
| 8 | +- Install the tracer module and import helper utilities for running traced Python snippets. |
| 9 | +- Provide fixtures that clear the trace buffer and reset global state between tests. |
| 10 | + |
| 11 | +## Tool Initialization |
| 12 | +- Acquire a monitoring tool ID and ensure subsequent calls reuse the same identifier. |
| 13 | +- Register callbacks for all enabled events and verify the resulting mask matches the design. |
| 14 | +- Unregister callbacks on shutdown and confirm no events fire afterward. |
| 15 | + |
| 16 | +## Event Recording |
| 17 | +### Control Flow Events |
| 18 | +- Capture `PY_START` and `PY_RETURN` for a simple script and assert a start/stop pair is recorded. |
| 19 | +- Resume and yield events within a generator function produce matching `PY_RESUME`/`PY_YIELD` entries. |
| 20 | +- A `PY_THROW` followed by `RERAISE` generates the expected unwind and rethrow sequence. |
| 21 | + |
| 22 | +### Call Tracking |
| 23 | +- Direct function calls record `CALL` and `PY_RETURN` with correct frame identifiers. |
| 24 | +- Recursive calls nest frames correctly and unwind in LIFO order. |
| 25 | +- Decorated functions ensure wrapper frames are recorded separately from wrapped frames. |
| 26 | + |
| 27 | +### Line and Branch Coverage |
| 28 | +- A loop with conditional branches emits `LINE` events for each executed line and `BRANCH` for each branch taken or skipped. |
| 29 | +- Jump statements such as `continue` and `break` produce `JUMP` events with source and destination line numbers. |
| 30 | + |
| 31 | +### Exception Handling |
| 32 | +- Raising and catching an exception emits `RAISE` and `EXCEPTION_HANDLED` events with matching exception IDs. |
| 33 | +- An uncaught exception records `RAISE` followed by `PY_UNWIND` and terminates the trace with a `PY_THROW`. |
| 34 | + |
| 35 | +### C API Boundary |
| 36 | +- Calling a built-in like `len` results in `C_CALL` and `C_RETURN` events linked to the Python frame. |
| 37 | +- A built-in that raises, such as `int("a")`, generates `C_RAISE` with the translated exception value. |
| 38 | + |
| 39 | +## Value Translation |
| 40 | +- Primitive values (ints, floats, strings, bytes) round-trip through the value registry and appear in the trace as expected. |
| 41 | +- Complex collections like lists of dicts are serialized recursively with cycle detection preventing infinite loops. |
| 42 | +- Object references without safe representations fall back to `repr` with a stable identifier. |
| 43 | + |
| 44 | +## Metadata and Source Capture |
| 45 | +- The trace writer copies the executing script into the output directory and records its SHA-256 hash. |
| 46 | +- Traces include `ProcessMetadata` fields for Python version and platform. |
| 47 | + |
| 48 | +## Shutdown Behavior |
| 49 | +- Normal interpreter exit flushes the trace and closes files without losing events. |
| 50 | +- An abrupt shutdown via `os._exit` truncates the trace file but leaves previous events intact. |
| 51 | + |
| 52 | +## Error and Edge Cases |
| 53 | +- Invalid event names in manual callback registration raise a clear `ValueError`. |
| 54 | +- Attempting to trace after the writer is closed results in a no-op without raising. |
| 55 | +- Large string values exceeding the configured limit are truncated with an explicit marker. |
| 56 | + |
| 57 | +## Performance and Stress |
| 58 | +- Tracing a tight loop of 10⁶ iterations completes within an acceptable time budget. |
| 59 | +- Concurrent threads each produce isolated traces with no frame ID collisions. |
| 60 | + |
0 commit comments