Enterprise-grade error handling and logging (#45)

Tried to centralize and manage error-handling and logging. Here's the
status of the task. See the README.md for better description of the new
features.


## WS1 – Foundations & Inventory
State: In progress
Tooling: `just errors-audit` (finds `PyRuntimeError::new_err`,
`unwrap`/`expect`/`panic!`, Python `RuntimeError`/`ValueError`).
What we saw:
- Rust modules now emit `RecorderError`; raw Python exceptions survive
in `codetracer_python_recorder/session.py` and tests (ISSUE-014).
- `src/monitoring/tracer.rs` still uses `lock().unwrap()` and lacks
error reporting for callback failures (ISSUE-013).
- Python glue keeps legacy assertions/unwraps (ISSUE-012).
Next moves:
- Land ISSUE-013 to sort the locking story.
- Plan the Python facade cleanup (ISSUE-014) once WS4 is steady.

## WS2 – `recorder-errors` Crate
State: Done (2025-10-02)
Highlights:
- Added `crates/recorder-errors` with `RecorderError`, enums, context
helpers, macros (`usage!`, `enverr!`, `target!`, `bug!`, `ensure_*`),
plus tests and optional serde support.
- `cargo test -p recorder-errors` + workspace `cargo check` stay green.
Next moves: Use this crate everywhere in WS3/WS4 work.

## WS3 – Retrofit Rust Modules
State: Done (2025-10-02)
Highlights:
- `session/*`, `runtime/*`, and `monitoring/tracer.rs` now return
`RecorderError` via the shared macros.
- Python exposure happens through one `errors` mapper; IO errors now
carry context.
- No stray `PyRuntimeError::new_err` left outside that mapper.
Next moves: Feed findings into WS4 and loop back to WS1 issues.

## WS4 – FFI Wrapper & Python Exception Hierarchy
State: Done (2025-10-02)
Highlights:
- Added `ffi` guard around each PyO3 entry point to map `RecorderError`
plus panic safety.
- Exposed Python classes `RecorderError`, `UsageError`,
`EnvironmentError`, `TargetError`, `InternalError`.
- Rust and Python tests cover the new flow (`uv run cargo nextest run
...`; `.venv/bin/python -m pytest ...`).
Next moves: Hold for WS5 until ISSUES 013/014 close.

## WS5 – Policy Switches & Runtime Configuration
State: Done (2025-10-03)
Highlights:
- `TraceSession.start()` and `trace()` now refresh policy from env vars
and accept override mappings so embeds wire recorder switches without
manual plumbing.
- Rust exports expose `configure_policy`/`configure_policy_from_env`
under the expected Python names; unit tests cover env-driven and
explicit override flows.
- Runtime tracer finish path honours `RecorderPolicy`: callback errors
respect `on_recorder_error` (disable detaches without surfacing
exceptions), `require_trace` now fails cleanly when no events land, and
partial traces are deleted or retained based on `keep_partial_trace`.
- Python CLI integration tests exercise disable vs abort paths and
require-trace enforcement using the new failure-injection toggles; CLI
now propagates runtime shutdown errors so exit codes reflect policy
outcomes while partial traces are cleaned per configuration.
Next moves: Kick off WS6 once upstream WS1 cleanups land.

## WS6 – Logging, Metrics, and Diagnostics
State: Done (2025-10-03)
Highlights:
- Replaced the `env_logger` helper with a structured JSON logger that
always emits `run_id`, active `trace_id`, and `error_code` fields while
honouring policy-driven log level and log file overrides.
- Introduced a pluggable `RecorderMetrics` sink and instrumented dropped
locations, policy-triggered detachments, and caught panics across the
monitoring/runtime paths; Rust unit tests exercise the metrics capture.
- Enabled the `--json-errors` policy path so runtime shutdown emits a
single-line JSON trailer on stderr; CLI integration tests now assert the
abort flow surfaces the trailer alongside existing stack traces.
Next moves: Wire the metrics sink into the chosen exporter and align the
log schema with Observability consumption before rolling out to
downstream tooling.

## WS7 – Test Coverage & Tooling Enforcement
State: Done (2025-10-04)
Highlights:
- Expanded `recorder-errors` and policy unit tests covering every macro
(usage/target/internal ensures) plus invalid boolean parsing.
- Added FFI unit tests for `dispatch`/`wrap_pyfunction`, panic
containment, and Python exception attribute propagation.
- Introduced integration coverage for environment permission failures,
injected target argument capture errors, and synthetic callback panics
(verifying JSON trailers and error classes).
- Implemented `just lint` orchestration running `cargo clippy -D
clippy::panic` and a repository script that blocks unchecked `.unwrap(`
usage outside the legacy allowlist.
Next moves: Monitor unwrap allowlist shrinkage once WS1 follow-ups land;
evaluate extending the lint to `.expect(` once monitoring refactor
closes.

## WS8 – Documentation & Rollout
State: Done (2025-10-05)
Highlights:
- README now covers the recorder error policy, JSON trailers, exit
codes, and a short Python `RecorderError` catch example.
- Added `docs/onboarding/error-handling.md` with migration steps, policy
wiring tips, and assertion rules for contributors.
- Started `codetracer-python-recorder/CHANGELOG.md` to brief downstream
tools on consuming structured errors.
Next moves:
- Share the onboarding doc with downstream maintainers and collect gaps
before promoting ADR 0004 to **Accepted**.
- Fold feedback into the change log before the next release tag.

Author: tzanko-matev

.coverage

@@ -1 +1,3 @@
-use flake
+watch_file nix/flake.nix
+watch_file nix/flake.lock
+use flake ./nix

.envrc

@@ -23,16 +23,16 @@ jobs:
           extra_nix_config: |
             experimental-features = nix-command flakes
       - name: Prepare dev environment
-        run: nix develop --command bash -lc 'just venv ${{matrix.python-version}} dev'
+        run: nix develop ./nix --command bash -lc 'just venv ${{matrix.python-version}} dev'
 
       - name: Verify recorder version metadata
-        run: nix develop --command bash -lc 'python3 scripts/check_recorder_version.py'
+        run: nix develop ./nix --command bash -lc 'python3 scripts/check_recorder_version.py'
 
       - name: Rust tests
-        run: nix develop --command bash -lc 'just cargo-test'
+        run: nix develop ./nix --command bash -lc 'just cargo-test'
 
       - name: Python tests
-        run: nix develop --command bash -lc 'just py-test'
+        run: nix develop ./nix --command bash -lc 'just py-test'
 
   coverage:
     name: Coverage (Python 3.12)
@@ -52,17 +52,17 @@ jobs:
             experimental-features = nix-command flakes
 
       - name: Prepare dev environment (Python 3.12)
-        run: nix develop --command bash -lc 'just venv 3.12 dev'
+        run: nix develop ./nix --command bash -lc 'just venv 3.12 dev'
 
       - name: Collect coverage
         id: coverage-run
-        run: nix develop --command bash -lc 'just coverage'
+        run: nix develop ./nix --command bash -lc 'just coverage'
 
       - name: Generate coverage comment
         if: steps.coverage-run.outcome == 'success'
         run: |
           ROOT="$(pwd)"
-          nix develop --command bash -lc "python3 codetracer-python-recorder/scripts/generate_coverage_comment.py \
+          nix develop ./nix --command bash -lc "python3 codetracer-python-recorder/scripts/generate_coverage_comment.py \
             --rust-summary codetracer-python-recorder/target/coverage/rust/summary.json \
             --python-json codetracer-python-recorder/target/coverage/python/coverage.json \
             --output codetracer-python-recorder/target/coverage/coverage-comment.md \

.github/workflows/ci.yml

@@ -7,6 +7,6 @@ build
 *~
 .idea/
 .cargo/
-
+.coverage
 **/*.egg-info/
 codetracer-python-recorder/codetracer_python_recorder/*.so

.github/workflows/maturin-generated.yaml

@@ -31,22 +31,44 @@ venv version=PYTHON_DEFAULT_VERSION:
 
 # Build the module in dev mode
 dev:
-    uv run --directory codetracer-python-recorder maturin develop --uv
+    uv run --directory codetracer-python-recorder maturin develop --uv --features integration-test
 
 # Run unit tests of dev build
 test: cargo-test py-test
 
 # Run Rust unit tests without default features to link Python C library
 cargo-test:
-    uv run cargo nextest run --manifest-path codetracer-python-recorder/Cargo.toml --no-default-features
+    uv run cargo nextest run --manifest-path codetracer-python-recorder/Cargo.toml --workspace --no-default-features
 
 py-test:
     uv run --group dev --group test pytest codetracer-python-recorder/tests/python codetracer-pure-python-recorder
+
+lint: lint-rust lint-errors
+
+lint-rust:
+    uv run cargo clippy --manifest-path codetracer-python-recorder/Cargo.toml --workspace --no-default-features -- -D clippy::panic
+
+lint-errors:
+    uv run python3 codetracer-python-recorder/scripts/lint_no_unwraps.py
 
 # Run tests only on the pure recorder
 test-pure:
     uv run --group dev --group test pytest codetracer-pure-python-recorder
 
+# Inspect ad-hoc error handling patterns across the Rust/Python recorder
+errors-audit:
+    @echo "== PyRuntimeError construction =="
+    @rg --color=never --no-heading -n "PyRuntimeError::new_err" codetracer-python-recorder/src codetracer-python-recorder/tests codetracer-python-recorder/codetracer_python_recorder || true
+    @echo
+    @echo "== unwrap()/expect()/panic! usage =="
+    @rg --color=never --no-heading -n "\\.unwrap\\(" codetracer-python-recorder/src || true
+    @rg --color=never --no-heading -n "\\.expect\\(" codetracer-python-recorder/src || true
+    @rg --color=never --no-heading -n "panic!" codetracer-python-recorder/src || true
+    @echo
+    @echo "== Python-side bare RuntimeError/ValueError =="
+    @rg --color=never --no-heading -n "raise RuntimeError" codetracer-python-recorder/codetracer_python_recorder || true
+    @rg --color=never --no-heading -n "raise ValueError" codetracer-python-recorder/codetracer_python_recorder || true
+
 # Generate combined coverage artefacts for both crates
 coverage:
     just coverage-rust