Remove base64 encoding

codetracer-python-recorder/Cargo.lock: 
codetracer-python-recorder/Cargo.toml: 
codetracer-python-recorder/src/runtime/mod.rs: 
design-docs/adr/0008-line-aware-io-capture.md: 
design-docs/io-capture-line-proxy-implementation-plan.md: 

Signed-off-by: Tzanko Matev <[email protected]>

Author: tzanko-matev

codetracer-python-recorder/Cargo.lock

@@ -31,7 +31,6 @@ serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 uuid = { version = "1.10", features = ["v4"] }
 recorder-errors = { version = "0.1.0", path = "crates/recorder-errors" }
-base64 = "0.22"
 
 [dev-dependencies]
 pyo3 = { version = "0.25.1", features = ["auto-initialize"] }

codetracer-python-recorder/Cargo.toml

@@ -47,9 +47,6 @@ use crate::runtime::io_capture::{
     IoChunk, IoChunkConsumer, IoChunkFlags, IoEventSink, IoStream, IoStreamProxies, ProxySink,
     ScopedMuteIoCapture,
 };
-
-use base64::engine::general_purpose::STANDARD as BASE64;
-use base64::Engine;
 use serde::Serialize;
 use serde_json;
 
@@ -389,7 +386,7 @@ impl RuntimeTracer {
         };
 
         let metadata = self.build_io_metadata(&chunk);
-        let content = BASE64.encode(&chunk.payload);
+        let content = String::from_utf8_lossy(&chunk.payload).into_owned();
 
         TraceWriter::add_event(
             &mut self.writer,
@@ -814,8 +811,6 @@ mod tests {
     use super::*;
     use crate::monitoring::CallbackOutcome;
     use crate::policy;
-    use base64::engine::general_purpose::STANDARD as BASE64;
-    use base64::Engine;
     use pyo3::types::{PyAny, PyCode, PyModule};
     use pyo3::wrap_pyfunction;
     use runtime_tracing::{FullValueRecord, StepRecord, TraceLowLevelEvent, ValueRecord};
@@ -1112,8 +1107,7 @@ result = compute()\n"
                 .filter_map(|event| match event {
                     TraceLowLevelEvent::Event(record) => {
                         let metadata: IoMetadata = serde_json::from_str(&record.metadata).ok()?;
-                        let payload = BASE64.decode(&record.content).ok()?;
-                        Some((metadata, payload))
+                        Some((metadata, record.content.as_bytes().to_vec()))
                     }
                     _ => None,
                 })

codetracer-python-recorder/src/runtime/mod.rs

@@ -28,6 +28,7 @@
 7. Guard our own logging with a `ScopedMuteIoCapture` RAII helper. It sets a thread-local flag so proxy callbacks short-circuit when the recorder writes to stderr.
 8. Add an optional best-effort FD mirror for `stdout`/`stderr`. When enabled it duplicates the file descriptors and spawns a reader that only handles writes not seen by the proxies. We track bytes seen by proxies in a per-stream ledger that keeps a FIFO byte buffer plus a monotonic sequence ID. The mirror removes ledger bytes from every read chunk with a streaming diff: scan left to right, skip native bytes, and peel ledger entries whenever their bytes appear, even when native writes arrive first in the chunk. Whatever bytes remain become mirror-only output. The GIL keeps Python `write` calls serial, so proxy order matches the order the OS sees. When native code writes directly to the FD it appears in the diff as leftover bytes we record with a `FdMirror` source tag.
 9. Expose a policy flag `policy.io_capture.line_proxies` defaulting to `true`. The FD mirror stays off by default and hides behind `policy.io_capture.fd_fallback`.
+10. Encode captured payloads as raw UTF-8 strings when forwarding to `runtime_tracing`. We trim the old manual base64 layer so downstream tooling, including the Codetracer UI, can consume the bytes without a second decode pass. Non UTF-8 input falls back to lossless bytes from the mirror or replacement characters when proxies surface decoded text.
 
 ## Consequences
 - **Pros:** We align IO chunks with the current Python frame, match C extensions that honour `sys.stdout`, and keep console behaviour untouched. The design lives inside the existing lifecycle code.

design-docs/adr/0008-line-aware-io-capture.md

@@ -21,7 +21,7 @@ This plan replaces the old pipe-based capture plan. Sentences stay short for eas
 - Add `IoChunk` struct holding `{stream, payload, thread_id, snapshot, timestamp, flags}`.
 - `IoEventSink` groups writes by thread and stream. Batches flush when we hit newline, explicit flush, a Step boundary, or a 5 ms timer. Use `parking_lot::Mutex` and a `once_cell::sync::Lazy` timer wheel to keep locking simple.
 - Provide `flush_before_step(thread_id)` API. The monitoring callbacks call it right before they emit a Step event, then record the Step, then update the snapshot store. This enforces the `Step -> IO -> next Step` ordering.
-- Convert chunks into runtime trace events right after batching. Reuse existing encoders.
+- Convert chunks into runtime trace events right after batching. Feed raw payload bytes into `runtime_tracing` so consumers do not need to undo an extra base64 layer.
 - Integrate with the recorder error macros for faults (`usage!`, `ioerr!`).
 - Tests: unit tests for batching rules, timer flush, newline handling, guard on recursion, and the Step ordering API.
 - Exit: sink drops zero events during stress tests that flood stdout with short writes.