design: better module identifiers in call stack

design-docs/adr/0014-module-call-event-naming.md: 
design-docs/module-call-event-naming-implementation-plan.md: 

Signed-off-by: Tzanko Matev <[email protected]>

Author: tzanko-matev

design-docs/adr/0014-module-call-event-naming.md

@@ -0,0 +1,55 @@
+# ADR 0014: Module Call Events Report Actual Module Names
+
+- **Status:** Proposed
+- **Date:** 2025-10-26
+- **Deciders:** codetracer recorder maintainers
+- **Consulted:** DX/observability stakeholders, Python runtime SMEs
+- **Informed:** Support engineers, product analytics consumers
+
+## Context
+- Every Python import executes the target module object under the hood. The recorder hooks `PY_START` for those executions and emits a synthetic “function call” whose name currently comes from `code.co_qualname`.
+- For module-level code objects CPython hardcodes `co_name == co_qualname == "<module>"`, so our trace file shows dozens (or hundreds) of `<module>` activations with no indication of which package was imported.
+- Trace consumers (CLI visualisations, query tools, and data-engineering pipelines) rely on the recorded function name to surface hot paths, build flame graphs, and attribute time/cost to particular packages. Without module names, users cannot tell whether a slow import belongs to `boto3`, `_distutils_hack`, or their own modules.
+- The trace filter engine already resolves module names from filenames / `sys.modules` to power `pkg:*` selectors, but the runtime tracer never reuses that information when assigning `FunctionId`s.
+
+## Problem
+- Module import events are indistinguishable in the trace log, making it impossible to attribute import costs, filter specific packages after the fact, or answer “which module executed here?” without cross-referencing filenames manually.
+- Because traces only show `<module>`, downstream tools collapse all module-level activations into a single node, hiding per-package behaviour and producing misleading metrics.
+- Purely using filenames as a proxy would leak physical layouts (e.g., `/usr/lib/python3.12/site-packages/boto3/__init__.py`) into the user-facing name and fails for zip apps or namespace packages.
+
+## Decision
+1. **Introduce a shared module identity helper.**
+   - Extract the existing module-derivation logic (`module_name_from_roots`, `lookup_module_name`) into a reusable service that can run independent of the filter engine.
+   - Accept a `CodeObjectWrapper` + `Python<'_>` handle and return a cached module name keyed by (code id, canonical filename). The helper first consults the filter resolution (if available), then repeats the relative-path inference, and finally falls back to `sys.modules` + frame globals (`__name__`) before conceding.
+2. **Rewrite `<module>` names as `<{resolved_name}>`.**
+   - When `RuntimeTracer::ensure_function_id` observes `co_qualname == "<module>"`, it queries the helper for the semantic module name.
+   - If a valid dotted identifier comes back (e.g., `boto3`, `foo.bar`), the tracer registers the function as `<boto3>` / `<foo.bar>` so downstream tooling still recognises the syntactic angle-bracket convention while learning the package involved.
+   - If resolution fails or yields garbage (non-identifier characters), the tracer keeps the legacy `<module>` label to avoid fabricating bad data.
+3. **Cache aggressively and keep the hot path cheap.**
+   - Store successful and failed lookups in `HashMap<usize, Option<String>>` keyed by code id so we only touch `sys.modules` or frame globals once per module.
+   - Reuse the helper from both the runtime tracer and (later) the pure-Python recorder to ensure consistent naming semantics across products.
+4. **Document the behaviour.**
+   - Update the recorder docs to state that module-level call events now show `<module-name>` and call out the limited cases (synthetic filenames, frozen modules, namespace packages) where the fallback remains `<module>`.
+
+## Consequences
+- **Pros**
+  - Import-heavy traces become readable: users immediately know which modules executed without digging through file paths.
+  - Post-processing / analytics pipelines gain a stable key (the dotted module name) to aggregate import costs or identify slow third-party packages.
+  - Reusing the existing derivation logic keeps behaviour aligned with trace filters and avoids duplicating heuristics.
+- **Cons**
+  - The tracer must occasionally scan `sys.modules` or inspect frame globals, which would add a small amount of work the first time we see each module. Caching mitigates the steady-state cost.
+  - Changing the function label alters the textual output of existing traces, so snapshot tests or downstream expectations that explicitly compare against `<module>` need updates.
+- **Risks**
+  - Mis-resolving a module (e.g., due to namespace packages exposing multiple files) could misattribute work to the wrong name. We guard this by preferring explicit `__spec__.name` / `__name__` over path guesses and falling back to `<module>` when ambiguous.
+  - Frames without real filenames (`<string>`, `<frozen ...>`) still cannot produce a meaningful module name. We explicitly document this to prevent support churn.
+  - Accessing `sys.modules` must hold the GIL and avoid long-lived Python references; the helper API enforces that discipline.
+
+## Alternatives
+- **Keep `<module>` and rely on filenames elsewhere.** Rejected because the filename is not present on every trace consumer surface and is cumbersome for humans.
+- **Rewrite module names from filenames only.** Rejected due to incorrect results for namespace packages, zip imports, site-packages bytecode caches, and `.pyc` vs `.py` mismatches.
+- **Add a new event field for module name.** Rejected to avoid changing the trace file schema; reusing the existing `function_name` field keeps compat with all tooling.
+
+## References
+- `codetracer-python-recorder/src/runtime/tracer/runtime_tracer.rs` (`ensure_function_id`).
+- `codetracer-python-recorder/src/runtime/tracer/events.rs` (`register_call_record`).
+- `codetracer-python-recorder/src/trace_filter/engine.rs` (current module name derivation & caching).

design-docs/module-call-event-naming-implementation-plan.md

@@ -0,0 +1,70 @@
+# Module Call Event Naming – Implementation Plan
+
+Plan owners: codetracer recorder maintainers  
+Target ADR: 0014 – Module Call Events Report Actual Module Names  
+Impacted components: `codetracer-python-recorder/src/runtime/tracer`, `src/runtime/value_capture`, `src/trace_filter`, design docs & user docs
+
+## Goals
+- Replace the `<module>` placeholder emitted during module imports with the actual dotted module name (e.g., `<boto3>`, `<foo.bar>`).
+- Share a single, cached module identity resolver between trace filtering and runtime event recording so both systems agree on module names.
+- Keep the hot path fast by caching module resolutions per code object and falling back to `<module>` only when we genuinely cannot determine the name.
+- Provide regression tests proving the trace writer records the new labels for in-project modules, site-packages imports, and namespace packages with real filenames.
+
+## Non-Goals
+- Changing the trace file schema or emitting additional fields beyond the function name.
+- Reworking how module names feed into configurable trace filters (that behaviour is covered by ADR 0013).
+- Retrofitting the pure-Python recorder in this iteration, though the helper should make that future work straightforward.
+
+## Current Gaps
+- `RuntimeTracer::ensure_function_id` always uses `code.co_qualname`, which is `<module>` for every top-level module execution.
+- Module name derivation logic lives inside `trace_filter::engine` and cannot be reused without duplicating heuristics (relative path stripping, `sys.modules` scan, `.pyc` awareness).
+- There is no cache tying `code_id` → module name outside the filter cache, so even if we bolted on ad-hoc lookups we would keep rescanning `sys.modules`.
+- No tests assert the textual function name for import events, so regressions would go unnoticed.
+
+## Workstreams
+
+### WS1 – Shared Module Identity Helper
+**Scope:** Factor reusable module name derivation ahead of tracer integration.
+- Extract `module_name_from_roots`, `lookup_module_name`, normalization helpers, and identifier validation from `trace_filter::engine` into a new module (e.g., `module_identity.rs`) under `src/common` or similar.
+- Add APIs:
+  - `ModuleIdentityResolver::from_sys_path(py) -> Self` to snapshot `sys.path` roots.
+  - `fn resolve_for_code(&self, py, code, frame_globals_name: Option<&str>) -> Option<String>` that performs (1) cached lookup by code id, (2) relative path inference, (3) `sys.modules` scan, and (4) optional `__name__` override when provided.
+- Allow the resolver to accept already-known module names (from `ScopeResolution.module_name`) so we do not recompute them.
+- Write focused Rust unit tests that stub out `sys.modules` entries to cover `.py` vs `.pyc` matches, namespace packages (`package/__init__.py`), and failure cases.
+- Update `trace_filter::engine` to depend on the new helper rather than its private copy, keeping behaviour identical.
+
+### WS2 – Runtime Tracer Integration
+**Scope:** Use the helper to rename `<module>` call events.
+- Extend `RuntimeTracer` with a `module_name_cache: HashMap<usize, Option<String>>` (or wrap the helper inside the tracer) and ensure it is cleared when the tracer resets.
+- When registering a call:
+  - Check whether the filter resolution already cached a module name; pass it into the resolver to avoid recomputation.
+  - Detect module-level code by checking `qualname == "<module>"` (and guard against variants like `"<module>"` with whitespace).
+  - Ask the helper for a module name; when successful, format the function label as `format!("<{name}>")` before calling `TraceWriter::ensure_function_id`.
+  - If the helper cannot decide, continue emitting `<module>` to preserve backwards compatibility.
+- Consider inspecting the captured frame snapshot (already obtained for argument capture) to pull `globals["__name__"]` as an extra hint when `sys.modules` fails; plumb that optional string into the helper to keep the `ensure_function_id` signature narrow.
+- Emit debug logs when we fall back to `<module>` despite having a real filename so troubleshooting remains possible.
+
+### WS3 – Testing, Tooling, and Docs
+**Scope:** Prove the behaviour change and explain it to users.
+- Add Python integration tests (e.g., `tests/python/test_monitoring_events.py`) that:
+  - Import a first-party module (`tests/python/support/sample_pkg/__init__.py`) and assert the trace JSON contains `<tests.python.support.sample_pkg>`.
+  - Import a third-party-like package placed under a temporary directory inserted into `sys.path` to ensure `sys.modules` + relative path fallback works.
+  - Cover namespace packages or packages that only expose `__spec__.name` to ensure we trust metadata over filesystem guesses.
+- Add Rust tests for the resolver cache (verify we only touch `sys.modules` once per module) and for the formatting logic in `ensure_function_id`.
+- Update `docs/README.md` (or recorder-specific docs) to mention that module names now appear in angle brackets, along with any troubleshooting guidance for cases where `<module>` persists (synthetic filenames, frozen imports).
+- Refresh changelog entries and any CLI snapshot tests that checked for `<module>`.
+
+## Testing Strategy
+- `cargo test -p codetracer-python-recorder` to run the new resolver unit tests.
+- `just test` to exercise Python integration tests and ensure traces serialize as expected.
+- Manual verification script that traces `python - <<'PY' ...` importing `boto3` (or a stub) and inspects the `.trace` file with `runtime_tracing::TraceReader`.
+
+## Risks & Mitigations
+- **Performance regressions:** Taking the GIL and scanning `sys.modules` per module could add overhead. Mitigate via per-code caches and by reusing filter-derived names when available.
+- **Incorrect attribution:** Namespace packages or reloaded modules might map multiple files to one name. Mitigate by preferring `__spec__.name`/`__name__` and logging whenever multiple candidates compete.
+- **Tight coupling with filters:** If the helper accidentally references filter-specific types, it becomes impossible to reuse elsewhere. Keep the helper independent and inject filter results as plain `Option<String>`.
+- **Test brittleness:** Snapshot tests referencing `<module>` need refreshing; add helper assertions that look for `<...>` pattern rather than hard-coded `<module>`.
+
+## Open Questions
+- Should we expose the resolved module name anywhere else (e.g., as metadata on trace records) to aid scripting? For now we only change the function label, but we might want to surface the raw dotted name later.
+- How should we treat modules executed via `runpy.run_module` with `__name__ == "__main__"` but living under a package path? The helper will return `"__main__"`; confirm that is acceptable or consider using the package dotted path derived from filename when `__name__ == "__main__"`.