Skip to content

Commit b2e9e09

Browse files
committed
Add Python tracer test design
1 parent 7fa3748 commit b2e9e09

File tree

1 file changed

+60
-0
lines changed

1 file changed

+60
-0
lines changed

docs/py-test-design-001.md

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Python sys.monitoring Tracer Test Design
2+
3+
## Overview
4+
This document outlines a test suite for validating the Python tracer built on `sys.monitoring` and `runtime_tracing`. Each test item corresponds to roughly 1–10 lines of implementation and exercises tracer behavior under typical and edge conditions.
5+
6+
## Setup
7+
- Establish a temporary directory for trace output and source snapshots.
8+
- Install the tracer module and import helper utilities for running traced Python snippets.
9+
- Provide fixtures that clear the trace buffer and reset global state between tests.
10+
11+
## Tool Initialization
12+
- Acquire a monitoring tool ID and ensure subsequent calls reuse the same identifier.
13+
- Register callbacks for all enabled events and verify the resulting mask matches the design.
14+
- Unregister callbacks on shutdown and confirm no events fire afterward.
15+
16+
## Event Recording
17+
### Control Flow Events
18+
- Capture `PY_START` and `PY_RETURN` for a simple script and assert a start/stop pair is recorded.
19+
- Resume and yield events within a generator function produce matching `PY_RESUME`/`PY_YIELD` entries.
20+
- A `PY_THROW` followed by `RERAISE` generates the expected unwind and rethrow sequence.
21+
22+
### Call Tracking
23+
- Direct function calls record `CALL` and `PY_RETURN` with correct frame identifiers.
24+
- Recursive calls nest frames correctly and unwind in LIFO order.
25+
- Decorated functions ensure wrapper frames are recorded separately from wrapped frames.
26+
27+
### Line and Branch Coverage
28+
- A loop with conditional branches emits `LINE` events for each executed line and `BRANCH` for each branch taken or skipped.
29+
- Jump statements such as `continue` and `break` produce `JUMP` events with source and destination line numbers.
30+
31+
### Exception Handling
32+
- Raising and catching an exception emits `RAISE` and `EXCEPTION_HANDLED` events with matching exception IDs.
33+
- An uncaught exception records `RAISE` followed by `PY_UNWIND` and terminates the trace with a `PY_THROW`.
34+
35+
### C API Boundary
36+
- Calling a built-in like `len` results in `C_CALL` and `C_RETURN` events linked to the Python frame.
37+
- A built-in that raises, such as `int("a")`, generates `C_RAISE` with the translated exception value.
38+
39+
## Value Translation
40+
- Primitive values (ints, floats, strings, bytes) round-trip through the value registry and appear in the trace as expected.
41+
- Complex collections like lists of dicts are serialized recursively with cycle detection preventing infinite loops.
42+
- Object references without safe representations fall back to `repr` with a stable identifier.
43+
44+
## Metadata and Source Capture
45+
- The trace writer copies the executing script into the output directory and records its SHA-256 hash.
46+
- Traces include `ProcessMetadata` fields for Python version and platform.
47+
48+
## Shutdown Behavior
49+
- Normal interpreter exit flushes the trace and closes files without losing events.
50+
- An abrupt shutdown via `os._exit` truncates the trace file but leaves previous events intact.
51+
52+
## Error and Edge Cases
53+
- Invalid event names in manual callback registration raise a clear `ValueError`.
54+
- Attempting to trace after the writer is closed results in a no-op without raising.
55+
- Large string values exceeding the configured limit are truncated with an explicit marker.
56+
57+
## Performance and Stress
58+
- Tracing a tight loop of 10⁶ iterations completes within an acceptable time budget.
59+
- Concurrent threads each produce isolated traces with no frame ID collisions.
60+

0 commit comments

Comments
 (0)