Add PM-facing user stories for error handling rollout

.agents/tasks/2025/08/21-0939-codetype-interface

@@ -40,3 +40,6 @@ Implement the CodeObjectWrapper as designed. Update the Tracer trait as well as
 There is an issue in the current implementation. We don't use caching effectively, since we create a new CodeObjectWrapper at each callback_xxx call. We need a global cache, probably keyed by the code object id. Propose design changes and update the design documents. Don't implement the changes themselves before I approve them.
 --- FOLLOW UP TASK ---
 Implement the global code object registry.
+
+--- FOLLOW UP TASK ---
+In this branch we have implemented standardized error-handling and logging. But we didn't get those features approved by the PM. Look at the changes, also at the design documents for the error handling and write down a set of user stories to give to the PM

.envrc

@@ -1 +1,3 @@
-use flake
+watch_file nix/flake.nix
+watch_file nix/flake.lock
+use flake ./nix

.github/workflows/ci.yml

@@ -23,13 +23,13 @@ jobs:
           extra_nix_config: |
             experimental-features = nix-command flakes
       - name: Prepare dev environment
-        run: nix develop --command bash -lc 'just venv ${{matrix.python-version}} dev'
+        run: nix develop ./nix --command bash -lc 'just venv ${{matrix.python-version}} dev'
 
       - name: Rust tests
-        run: nix develop --command bash -lc 'just cargo-test'
+        run: nix develop ./nix --command bash -lc 'just cargo-test'
 
       - name: Python tests
-        run: nix develop --command bash -lc 'just py-test'
+        run: nix develop ./nix --command bash -lc 'just py-test'
 
   coverage:
     name: Coverage (Python 3.12)
@@ -49,17 +49,17 @@ jobs:
             experimental-features = nix-command flakes
 
       - name: Prepare dev environment (Python 3.12)
-        run: nix develop --command bash -lc 'just venv 3.12 dev'
+        run: nix develop ./nix --command bash -lc 'just venv 3.12 dev'
 
       - name: Collect coverage
         id: coverage-run
-        run: nix develop --command bash -lc 'just coverage'
+        run: nix develop ./nix --command bash -lc 'just coverage'
 
       - name: Generate coverage comment
         if: steps.coverage-run.outcome == 'success'
         run: |
           ROOT="$(pwd)"
-          nix develop --command bash -lc "python3 codetracer-python-recorder/scripts/generate_coverage_comment.py \
+          nix develop ./nix --command bash -lc "python3 codetracer-python-recorder/scripts/generate_coverage_comment.py \
             --rust-summary codetracer-python-recorder/target/coverage/rust/summary.json \
             --python-json codetracer-python-recorder/target/coverage/python/coverage.json \
             --output codetracer-python-recorder/target/coverage/coverage-comment.md \

.gitignore

@@ -7,5 +7,5 @@ build
 *~
 .idea/
 .cargo/
-
+.coverage
 **/*.egg-info/

Justfile

@@ -31,22 +31,44 @@ venv version=PYTHON_DEFAULT_VERSION:
 
 # Build the module in dev mode
 dev:
-    uv run --directory codetracer-python-recorder maturin develop --uv
+    uv run --directory codetracer-python-recorder maturin develop --uv --features integration-test
 
 # Run unit tests of dev build
 test: cargo-test py-test
 
 # Run Rust unit tests without default features to link Python C library
 cargo-test:
-    uv run cargo nextest run --manifest-path codetracer-python-recorder/Cargo.toml --no-default-features
+    uv run cargo nextest run --manifest-path codetracer-python-recorder/Cargo.toml --workspace --no-default-features
 
 py-test:
     uv run --group dev --group test pytest codetracer-python-recorder/tests/python codetracer-pure-python-recorder
+
+lint: lint-rust lint-errors
+
+lint-rust:
+    uv run cargo clippy --manifest-path codetracer-python-recorder/Cargo.toml --workspace --no-default-features -- -D clippy::panic
+
+lint-errors:
+    uv run python3 codetracer-python-recorder/scripts/lint_no_unwraps.py
 
 # Run tests only on the pure recorder
 test-pure:
     uv run --group dev --group test pytest codetracer-pure-python-recorder
 
+# Inspect ad-hoc error handling patterns across the Rust/Python recorder
+errors-audit:
+    @echo "== PyRuntimeError construction =="
+    @rg --color=never --no-heading -n "PyRuntimeError::new_err" codetracer-python-recorder/src codetracer-python-recorder/tests codetracer-python-recorder/codetracer_python_recorder || true
+    @echo
+    @echo "== unwrap()/expect()/panic! usage =="
+    @rg --color=never --no-heading -n "\\.unwrap\\(" codetracer-python-recorder/src || true
+    @rg --color=never --no-heading -n "\\.expect\\(" codetracer-python-recorder/src || true
+    @rg --color=never --no-heading -n "panic!" codetracer-python-recorder/src || true
+    @echo
+    @echo "== Python-side bare RuntimeError/ValueError =="
+    @rg --color=never --no-heading -n "raise RuntimeError" codetracer-python-recorder/codetracer_python_recorder || true
+    @rg --color=never --no-heading -n "raise ValueError" codetracer-python-recorder/codetracer_python_recorder || true
+
 # Generate combined coverage artefacts for both crates
 coverage:
     just coverage-rust

README.md

@@ -2,11 +2,68 @@
 
 This repository now hosts two related projects:
 
-- codetracer-pure-python-recorder — the existing pure-Python prototype that records [CodeTracer](https://github.com/metacraft-labs/CodeTracer) traces using sys.settrace.
-- codetracer-python-recorder — a new, Rust-backed Python extension module (PyO3) intended to provide a faster and more featureful recorder.
+- codetracer-pure-python-recorder — a pure-Python tracer that still mirrors the early prototype.
+- codetracer-python-recorder — a Rust-backed Python extension (PyO3 + maturin) with structured errors and tighter tooling.
 
-> [!WARNING]
-> Both projects are early-stage prototypes. Contributions and discussion are welcome!
+Both projects are still in motion. Expect breaking changes while we finish the error-handling rollout.
+
+### Structured errors (Rust-backed recorder)
+
+The Rust module wraps every failure in a `RecorderError` hierarchy that reaches Python with a stable `code`, a readable `kind`, and a `context` dict.
+
+- `UsageError` → bad input or calling pattern. Codes like `ERR_ALREADY_TRACING`.
+- `EnvironmentError` → IO or OS problems. Codes like `ERR_IO`.
+- `TargetError` → the traced program raised or refused inspection. Codes like `ERR_TRACE_INCOMPLETE`.
+- `InternalError` → a recorder bug or panic. Codes default to `ERR_UNKNOWN` unless classified.
+
+Quick catch example:
+
+```python
+from codetracer_python_recorder import RecorderError, start, stop
+
+try:
+    session = start("/tmp/trace", format="json")
+except RecorderError as err:
+    print(f"Recorder failed: {err.code}")
+    for key, value in err.context.items():
+        print(f"  {key}: {value}")
+else:
+    try:
+        ...  # run work here
+    finally:
+        session.flush()
+        stop()
+```
+
+All subclasses carry the same attributes, so existing handlers can migrate by catching `RecorderError` once and branching on `err.code` if needed.
+
+### CLI exit behaviour and JSON trailers
+
+`python -m codetracer_python_recorder` returns:
+
+- `0` when tracing and the target script succeed.
+- The script's own exit code when it calls `sys.exit()`.
+- `1` when a `RecorderError` bubbles out of startup or shutdown.
+- `2` when the CLI arguments are incomplete.
+
+Pass `--codetracer-json-errors` (or set the policy via `configure_policy(json_errors=True)`) to stream a one-line JSON trailer on stderr. The payload includes `run_id`, `trace_id`, `error_code`, `error_kind`, `message`, and the `context` map so downstream tooling can log failures without scraping text.
+
+### Migration checklist for downstream tools
+
+- Catch `RecorderError` (or a subclass) instead of `RuntimeError`.
+- Switch any string matching over to `err.code` values like `ERR_TRACE_DIR_CONFLICT`.
+- Expect structured log lines (JSON) on stderr. Use the `error_code` field instead of parsing text.
+- Opt in to JSON trailers for machine parsing and keep human output short.
+- Update policy wiring to use `configure_policy` / `policy_snapshot()` rather than hand-rolled env parsing.
+- Read `docs/onboarding/error-handling.md` for detailed migration steps and assertion rules.
+
+### Logging defaults
+
+The recorder now installs a JSON logger on first import. Logs include `run_id`, optional `trace_id`, and an `error_code` field when set.
+
+- Control the log filter with `RUST_LOG=target=level` (standard env syntax).
+- Override from Python with `configure_policy(log_level="info")` or `log_file=...` for file output.
+- Metrics counters record dropped events, detach reasons, and caught panics; plug your own sink via the Rust API when embedding.
 
 ### codetracer-pure-python-recorder
 
@@ -54,6 +111,20 @@ Basic workflow:
 
 The CI workflow mirrors these commands. Pull requests get an automated comment with the latest Rust/Python coverage tables and downloadable artefacts (`lcov.info`, `coverage.xml`, `coverage.json`).
 
+#### Debug logging
+
+Rust-side logging defaults to `warn` so test output stays readable. Export
+`RUST_LOG` when you need more detail:
+
+```bash
+RUST_LOG=codetracer_python_recorder=debug pytest \
+  codetracer-python-recorder/tests/python/unit/test_backend_exceptions.py -q
+```
+
+Any filter accepted by `env_logger` still works, so you can switch to
+`RUST_LOG=codetracer_python_recorder=info` or silence everything with
+`RUST_LOG=off`.
+
 ### Future directions
 
 The current Python support is an unfinished prototype. We can finish it. In the future, it may be expanded to function in a way to similar to the more complete implementations, e.g. [Noir](https://github.com/blocksense-network/noir/tree/blocksense/tooling/tracer).

codetracer-python-recorder/CHANGELOG.md

@@ -0,0 +1,6 @@
+# codetracer-python-recorder — Change Log
+
+## Unreleased
+- Documented the error-handling policy. README now lists the `RecorderError` hierarchy, policy hooks (`configure_policy`, JSON trailers), exit codes, and sample handlers so Python callers can consume structured failures.
+- Added an onboarding guide under `docs/onboarding/error-handling.md` with migration steps for downstream tools.
+- Recorded assertion guidance for contributors: prefer `bug!`/`ensure_internal!` over raw `panic!`/`.unwrap()` and keep `debug_assert!` paired with classified errors.