feat(forwarder): system prompt for session chat (#418)

[jonathaneoliver]

Summary

• Authors prompts/session_chat.md — the system prompt that drives every AI session-analysis interaction (overview, forensic, compare). Covers role, schema, tool docs, response style, mode hints, and failure-mode language.
• Adds a small mtime-cached loader so operators can edit the prompt live (no forwarder restart) and the next request picks it up.
• Sub-issue 6 of epic epic: AI-assisted session analysis (chat + Analyze + compare) #412. Closes #418.
• Stacked on feat(forwarder): llm_calls ledger + USD budget guards (#417) #429.

What's in the prompt

• Role + how sessions are recorded (session_id ⨯ play_id, 1Hz snapshots, HAR-style network_requests).
• Schema of session_snapshots + network_requests + llm_calls with the columns analyses actually reach for.
• Response-style rules — anchor every claim to mm:ss.ms timestamps (the dashboard auto-links these to seek), structure forensic answers as Observation / Mechanism / Evidence, comparison answers as Similarities / Differences / Hypotheses, no padding, no SQL in answers unless asked.
• Mode hints triggered by the focus-context preamble (Focus session_id / Focus range / Compare sessions).
• Tool-use guidance — start narrow, aggregate before materializing, errors-first.
• Failure-mode language so a confused or data-sparse model stops cleanly instead of hallucinating.

Loader behaviour

• SessionChatPrompt() reads /config/prompts/session_chat.md (overridable via SESSION_CHAT_PROMPT_PATH).
• PromptCache.Load re-reads only on mtime change — edit the file in production, next chat picks it up automatically.
• Built-in stub fallback keeps /api/session_chat working when the file is missing or malformed.
• Frontmatter parser handles standard --- delimiters; missing or unversioned frontmatter logs unversioned into the ledger.

Ledger correlation

llm_calls.prompt_version now carries the version from the loaded prompt's frontmatter, so A/B iterations on the prompt correlate with response quality (e.g. cost per turn, average iterations, error rate) over time.

Test plan

• go test ./... 67/67 pass.
• docker build clean; image contains /config/prompts/session_chat.md (8.5 KB).
• TestSessionChat_InjectsSystemPrompt — uses a unique marker string to verify the on-disk prompt body actually reaches the LLM as the first message.
• Frontmatter parsing covers: no frontmatter, with frontmatter, unterminated frontmatter, unversioned frontmatter.
• PromptCache covers: re-read on mtime change, hit cache when unchanged, missing-file fallback to builtin.
• Live-network smoke against a real LLM with the new prompt — manual / deploy-time.

🤖 Generated with Claude Code

[jonathaneoliver]

Closing — superseded by the v2.0.0 release.

• The AI-chat backend (analytics: ClickHouse llm_reader user (read-only, hard server-side limits) #413–prompts: initial session_chat.md (schema + tool docs + citation rules) #418) was reimplemented and expanded directly on main: analytics/go-forwarder/llm_client.go, llm_session_chat.go, llm_profiles.go, llm_tool_query.go, llm_ledger.go, prompts/chat.md, plus the tiered tools + Anthropic client + chat persistence that didn't exist in this stack.
• The dashboard pieces (dashboard: Discuss panel + Analyze button + model picker on session-viewer.html / sessions.html #419–dashboard: two-session AI compare from sessions.html multi-select #421) targeted the legacy content/dashboard/ + content/shared/ surface, which the Vue 3 dashboard rewrite removed. Their diffs no longer apply to any file on main.

This stale stacked-PR chain (epic #412) has no salvageable diff against current main. The feature lives on in the v3 dashboard + forwarder; further work should target those, not this stack.