Io capture (#46)

Author: tzanko-matev

README.md

@@ -48,6 +48,36 @@ All subclasses carry the same attributes, so existing handlers can migrate by ca
 
 Pass `--codetracer-json-errors` (or set the policy via `configure_policy(json_errors=True)`) to stream a one-line JSON trailer on stderr. The payload includes `run_id`, `trace_id`, `error_code`, `error_kind`, `message`, and the `context` map so downstream tooling can log failures without scraping text.
 
+### IO capture configuration
+
+Line-aware capture (see [ADR 0008](design-docs/adr/0008-line-aware-io-capture.md)) installs `LineAwareStdout`, `LineAwareStderr`, and `LineAwareStdin` proxies so every chunk carries `{path_id, line, frame_id}` metadata. The proxies forward writes immediately to keep TTY behaviour unchanged and the batching sink emits newline/flush/step-delimited chunks. When the FD mirror fallback observes bytes that bypassed the proxies, the resulting `IoChunk` carries the `mirror` flag so downstream tooling can highlight native writers separately. Recorder logs and telemetry use `ScopedMuteIoCapture` to avoid recursive capture.
+
+Control the feature through the policy layer:
+
+- CLI: `python -m codetracer_python_recorder --io-capture=off script.py` disables capture, while `--io-capture=proxies+fd` also mirrors raw file-descriptor writes.
+- Python: `configure_policy(io_capture_line_proxies=False)` toggles proxies, and `configure_policy(io_capture_fd_fallback=True)` enables the FD fallback.
+- Environment: set `CODETRACER_CAPTURE_IO=off`, `proxies`, or `proxies+fd` (`,` is also accepted) to match the CLI and Python helpers.
+
+Manual smoke check: `python -m codetracer_python_recorder examples/stdout_script.py` should report the proxied output while leaving the console live.
+
+#### Troubleshooting replaced stdout/stderr
+
+Third-party tooling occasionally replaces `sys.stdout` / `sys.stderr` after the proxies install. When that happens, IO metadata stops updating and the recorder falls back to passthrough behaviour. You can verify the binding at runtime:
+
+```python
+import sys
+from codetracer_python_recorder.runtime import LineAwareStdout, LineAwareStderr
+
+print(type(sys.stdout).__name__, isinstance(sys.stdout, LineAwareStdout))
+print(type(sys.stderr).__name__, isinstance(sys.stderr, LineAwareStderr))
+```
+
+Both `isinstance` checks should return `True`. If they do not:
+
+1. Re-run `configure_policy(io_capture_line_proxies=True)` (or restart tracing) to reinstall the proxies before the other tool mutates the streams.
+2. Fall back to FD mirroring by enabling `CODETRACER_CAPTURE_IO=proxies+fd` so native writes still reach the ledger-backed mirror.
+3. As a last resort, disable IO capture (`--io-capture=off`) and rely on console output while investigating the conflicting integration.
+
 ### Migration checklist for downstream tools
 
 - Catch `RecorderError` (or a subclass) instead of `RuntimeError`.

codetracer-python-recorder/CHANGELOG.md

@@ -6,6 +6,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
 
 ## [Unreleased]
 ### Added
+- Introduced a line-aware IO capture pipeline that records stdout/stderr chunks with `{path_id, line, frame_id}` attribution via the shared `LineSnapshotStore` and multi-threaded `IoEventSink`.
+- Added `LineAwareStdout`, `LineAwareStderr`, and `LineAwareStdin` proxies that forward to the original streams while batching writes on newline, explicit `flush()`, 5 ms idle gaps, and step boundaries.
+- Added policy, CLI, and environment toggles for IO capture (`--io-capture`, `configure_policy(io_capture_line_proxies=..., io_capture_fd_fallback=...)`, `CODETRACER_CAPTURE_IO`) alongside the `ScopedMuteIoCapture` guard that suppresses recursive recorder logging.
+- Added an optional FD mirror fallback that duplicates `stdout`/`stderr`, diffs native writes against the proxy ledger, emits `mirror`-flagged `IoChunk`s, and restores descriptors on teardown.
+- Documented IO capture behaviour in the README with ADR 0008 context, manual smoke instructions, and troubleshooting steps for replaced `sys.stdout` / `sys.stderr`.
 - Documented the error-handling policy in the README, including the `RecorderError` hierarchy, policy hooks, JSON error trailers, exit codes, and sample handlers for structured failures.
 - Added an onboarding guide at `docs/onboarding/error-handling.md` with migration steps for downstream tools.
 - Added contributor guidance for assertions: prefer `bug!` / `ensure_internal!` over `panic!` / `.unwrap()`, and pair `debug_assert!` with classified errors.

codetracer-python-recorder/Cargo.lock

@@ -31,6 +31,7 @@ serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 uuid = { version = "1.10", features = ["v4"] }
 recorder-errors = { version = "0.1.0", path = "crates/recorder-errors" }
+libc = "0.2"
 
 [dev-dependencies]
 pyo3 = { version = "0.25.1", features = ["auto-initialize"] }

codetracer-python-recorder/Cargo.toml

@@ -97,6 +97,15 @@ def _parse_args(argv: Sequence[str]) -> RecorderCLIConfig:
         action="store_true",
         help="Emit JSON error trailers on stderr.",
     )
+    parser.add_argument(
+        "--io-capture",
+        choices=["off", "proxies", "proxies+fd"],
+        help=(
+            "Control stdout/stderr capture. Without this flag, line-aware proxies stay enabled. "
+            "'off' disables capture, 'proxies' forces proxies without FD mirroring, "
+            "'proxies+fd' also mirrors raw file-descriptor writes."
+        ),
+    )
 
     known, remainder = parser.parse_known_args(argv)
     pending: list[str] = list(remainder)
@@ -145,6 +154,19 @@ def _parse_args(argv: Sequence[str]) -> RecorderCLIConfig:
         policy["log_file"] = Path(known.log_file).expanduser().resolve()
     if known.json_errors:
         policy["json_errors"] = True
+    if known.io_capture:
+        match known.io_capture:
+            case "off":
+                policy["io_capture_line_proxies"] = False
+                policy["io_capture_fd_fallback"] = False
+            case "proxies":
+                policy["io_capture_line_proxies"] = True
+                policy["io_capture_fd_fallback"] = False
+            case "proxies+fd":
+                policy["io_capture_line_proxies"] = True
+                policy["io_capture_fd_fallback"] = True
+            case other:  # pragma: no cover - argparse choices block this
+                parser.error(f"unsupported io-capture mode '{other}'")
 
     return RecorderCLIConfig(
         trace_dir=trace_dir,

codetracer-python-recorder/codetracer_python_recorder/cli.py

@@ -145,12 +145,9 @@ mod tests {
     #[test]
     fn map_recorder_error_sets_python_attributes() {
         Python::with_gil(|py| {
-            let err = usage!(
-                ErrorCode::UnsupportedFormat,
-                "invalid trace format"
-            )
-            .with_context("format", "yaml")
-            .with_source(std::io::Error::new(std::io::ErrorKind::Other, "boom"));
+            let err = usage!(ErrorCode::UnsupportedFormat, "invalid trace format")
+                .with_context("format", "yaml")
+                .with_source(std::io::Error::new(std::io::ErrorKind::Other, "boom"));
             let pyerr = map_recorder_error(err);
             let ty = pyerr.get_type(py);
             assert!(ty.is(py.get_type::<PyUsageError>()));
@@ -191,9 +188,8 @@ mod tests {
     #[test]
     fn dispatch_converts_recorder_error_to_pyerr() {
         Python::with_gil(|py| {
-            let result: PyResult<()> = dispatch("dispatch_env", || {
-                Err(enverr!(ErrorCode::Io, "disk full"))
-            });
+            let result: PyResult<()> =
+                dispatch("dispatch_env", || Err(enverr!(ErrorCode::Io, "disk full")));
             let err = result.expect_err("expected PyErr");
             let ty = err.get_type(py);
             assert!(ty.is(py.get_type::<PyEnvironmentError>()));