Merge pull request #20 from beyond5959/dev

Dev

Author: beyond5959

PROGRESS.md

@@ -11,7 +11,24 @@ This file is the source of milestone progress, validation commands, and next act
 
 - `Post-M8` ACP multi-agent readiness and maintenance.
 
-## Latest Update (2026-03-09)
+## Latest Update (2026-03-11)
+
+- `Post-M8` codex session identity and replay normalization completed:
+  - fixed fresh Codex `New session` persistence so ngent no longer stores provisional runtime ids like `session-1` as the thread session binding when a durable `_meta.threadId` is not yet available.
+  - deferred initial `session_bound` persistence/emission for fresh Codex sessions until a stable session id can be resolved after the first prompt, then updated in-memory and persisted thread `agentOptions.sessionId` with the durable id.
+  - normalized Codex transcript replay by filtering bootstrap user messages injected by the desktop wrapper (`AGENTS.md` / `environment_context`) and extracting the actual user request from known wrapper formats:
+    - `[Conversation Summary] ... [Current User Input]`
+    - `# Context from my IDE setup: ... ## My request for Codex:`
+  - verified with real local Codex and Playwright against `http://127.0.0.1:8687/`:
+    - `New session` now produces distinct stable Codex session ids and no longer mixes first-session messages into the second-session chat.
+    - switching between the two replayed sessions in the Web UI now shows only the expected user/assistant pairs.
+    - direct `session-history` API responses for the repro sessions now return cleaned transcript messages.
+  - validation:
+    - pass: `go test ./internal/agents/codex -run 'Test(ParseSessionTranscriptMessage|CodexShouldDeferInitialSessionBinding|NormalizeCodexSessionListResultUsesStableThreadID|CodexSessionMatchesIDAcceptsStableAndRawIDs|CodexStableSessionIDFallsBackToRawSessionID)$' -count=1`
+    - pass: `cd internal/webui/web && npm run build`
+    - pass: `go test ./...`
+
+## Previous Update (2026-03-09)
 
 - Kimi CLI ACP integration completed:
   - implemented `internal/agents/kimi` with one-turn ACP stdio lifecycle and fail-closed permission handling.
@@ -574,3 +591,33 @@ This file is the source of milestone progress, validation commands, and next act
 - 2026-03-09: unified ACP message-chunk constant usage across stdio providers by removing per-provider `updateTypeMessageChunk` definitions and reusing `agents.ACPUpdateTypeMessageChunk`.
 - 2026-03-09: hid the Web UI Reasoning switch when the active agent exposes fewer than two reasoning choices, so agents without switchable reasoning no longer show a dead control.
 - 2026-03-09: switched Kimi model/reasoning catalog queries to local `config.toml` when available, so startup catalog refresh and thread config/model operations no longer create empty Kimi sessions; real prompt turns still use ACP.
+- 2026-03-11: added opt-in `--debug` startup flag; when enabled, stderr now emits sanitized `acp.message` traces for ACP stdio and embedded-runtime request/response traffic, including session prompts, updates, and permission flows.
+- 2026-03-11: added ACP session browsing/resume support across built-in agents:
+  - introduced shared agent session abstractions for `session/list`, bound-session reporting, and initialize capability parsing.
+  - built-in providers now:
+    - list sessions through ACP `session/list` when supported.
+    - load persisted `agentOptions.sessionId` through ACP `session/load` before prompting.
+    - report the effective session id back to HTTP turns so the server can persist it.
+  - added `GET /v1/threads/{threadId}/sessions` with cursor passthrough and graceful `supported=false` fallback for agents without ACP session-history support.
+  - turn SSE now emits `session_bound`, and the server persists the thread session id without closing the active provider.
+  - once a thread is bound to an ACP session, prompt building skips local recent-turn injection to avoid duplicating already-loaded ACP context.
+  - Web UI now renders a right-side session sidebar with first-page load, `Show more`, active-session highlighting, and `New session` reset.
+  - executed validation:
+    - pass: `cd internal/webui/web && npm run build`
+    - pass: `go test ./internal/httpapi -run 'TestThreadSessionsListEndpoint|TestTurnSessionBoundPersistsSessionIDAndSkipsContextInjection' -count=1`
+    - pass: `go test ./...`
+
+- 2026-03-11: fixed Web UI session playback when selecting an existing ACP session from the right sidebar.
+  - the active chat view now treats `(threadId, sessionId)` as its render scope instead of refreshing only on `threadId` changes.
+  - `loadHistory()` now filters locally persisted turns by each turn's `session_bound` event so the center chat panel replays the selected session's ngent-recorded turns instead of leaving the previous session on screen.
+  - session changes reported mid-stream by `session_bound` defer the full chat refresh until the active turn completes, so the live streaming bubble is not destroyed.
+  - executed validation:
+    - pass: `cd internal/webui/web && npm run build`
+    - pass: `go test ./...`
+
+- 2026-03-11: fixed Web UI history replay for legacy session threads whose `/history` data lacks per-turn `session_bound` events.
+  - session-scoped history filtering now falls back to showing all turns when a thread has no annotated session markers at all, instead of rendering an empty chat pane despite non-empty `/history`.
+  - when a thread has exactly one annotated session, the selected session view also keeps older unannotated turns so pre-annotation history is still visible for that same session.
+  - executed validation:
+    - pass: `cd internal/webui/web && npm run build`
+    - pass: `go test ./...`

cmd/ngent/main.go

@@ -46,6 +46,7 @@ func main() {
 
 	portFlag := flag.Int("port", 8686, "server listen port (1-65535)")
 	allowPublic := flag.Bool("allow-public", false, "allow listening on public interfaces (default false for loopback-only)")
+	debugFlag := flag.Bool("debug", false, "enable verbose debug logs, including ACP request/response payloads on stderr")
 	authToken := flag.String("auth-token", "", "optional bearer token for /v1/* endpoints")
 	dbPath := flag.String("db-path", defaultDBPath, "sqlite database path")
 	contextRecentTurns := flag.Int("context-recent-turns", 10, "number of recent user+assistant turns injected into each prompt")
@@ -55,6 +56,13 @@ func main() {
 	shutdownGraceTimeout := flag.Duration("shutdown-grace-timeout", 8*time.Second, "graceful shutdown timeout for active turns")
 	flag.Parse()
 
+	logLevel := slog.LevelInfo
+	if *debugFlag {
+		logLevel = slog.LevelDebug
+	}
+	logger = observability.NewJSONLogger(logLevel)
+	observability.ConfigureACPDebug(logger, *debugFlag)
+
 	codexRuntimeConfig := codexagent.DefaultRuntimeConfig()
 	codexPreflightErr := codexagent.Preflight(codexRuntimeConfig)
 	opencodePreflightErr := opencodeagent.Preflight()
@@ -108,6 +116,9 @@ func main() {
 	if claudePreflightErr != nil {
 		logger.Warn("startup.claude_unavailable", "error", claudePreflightErr.Error())
 	}
+	if *debugFlag {
+		logger.Info("startup.debug_enabled", "acpTrace", true)
+	}
 	agents := supportedAgents(codexAvailable, opencodeAvailable, geminiAvailable, kimiAvailable, qwenAvailable, claudeAvailable)
 
 	listenAddr, port, err := resolveListenAddr(*portFlag, *allowPublic)
@@ -148,12 +159,14 @@ func main() {
 		TurnController:  turnController,
 		TurnAgentFactory: func(thread storage.Thread) (agentimpl.Streamer, error) {
 			modelID := extractModelID(thread.AgentOptionsJSON)
+			sessionID := extractSessionID(thread.AgentOptionsJSON)
 			configOverrides := extractConfigOverrides(thread.AgentOptionsJSON)
 			switch thread.AgentID {
 			case "codex":
 				return codexagent.New(codexagent.Config{
 					Dir:             thread.CWD,
 					ModelID:         modelID,
+					SessionID:       sessionID,
 					ConfigOverrides: configOverrides,
 					Name:            "codex-embedded",
 					RuntimeConfig:   codexRuntimeConfig,
@@ -162,30 +175,35 @@ func main() {
 				return opencodeagent.New(opencodeagent.Config{
 					Dir:             thread.CWD,
 					ModelID:         modelID,
+					SessionID:       sessionID,
 					ConfigOverrides: configOverrides,
 				})
 			case "gemini":
 				return geminiagent.New(geminiagent.Config{
 					Dir:             thread.CWD,
 					ModelID:         modelID,
+					SessionID:       sessionID,
 					ConfigOverrides: configOverrides,
 				})
 			case "kimi":
 				return kimiagent.New(kimiagent.Config{
 					Dir:             thread.CWD,
 					ModelID:         modelID,
+					SessionID:       sessionID,
 					ConfigOverrides: configOverrides,
 				})
 			case "qwen":
 				return qwenagent.New(qwenagent.Config{
 					Dir:             thread.CWD,
 					ModelID:         modelID,
+					SessionID:       sessionID,
 					ConfigOverrides: configOverrides,
 				})
 			case "claude":
 				return claudeagent.New(claudeagent.Config{
 					Dir:             thread.CWD,
 					ModelID:         modelID,
+					SessionID:       sessionID,
 					ConfigOverrides: configOverrides,
 					Name:            "claude-embedded",
 				})
@@ -661,6 +679,19 @@ func extractModelID(agentOptionsJSON string) string {
 	return strings.TrimSpace(opts.ModelID)
 }
 
+func extractSessionID(agentOptionsJSON string) string {
+	var opts struct {
+		SessionID string `json:"sessionId"`
+	}
+	if strings.TrimSpace(agentOptionsJSON) == "" {
+		return ""
+	}
+	if err := json.Unmarshal([]byte(agentOptionsJSON), &opts); err != nil {
+		return ""
+	}
+	return strings.TrimSpace(opts.SessionID)
+}
+
 func extractConfigOverrides(agentOptionsJSON string) map[string]string {
 	var opts struct {
 		ConfigOverrides map[string]any `json:"configOverrides"`

docs/ACCEPTANCE.md

@@ -265,3 +265,42 @@ This checklist defines executable acceptance checks for requirements 1-16.
 - Verification commands (executed 2026-03-06):
   - `go test ./...`
   - `cd internal/webui/web && npm run build`
+
+## Requirement 22: ACP Debug Trace Logging
+
+- Operation:
+  - start server with `--debug=true`.
+  - execute an ACP-backed request path.
+  - inspect stderr logs.
+- Expected:
+  - logger runs at debug level.
+  - stderr includes `acp.message` entries for outbound and inbound ACP JSON-RPC traffic.
+  - entries include `component`, `direction`, `rpcType`, `method` when present, and sanitized `rpc` payload.
+  - sensitive fields/tokens are redacted before logging.
+- Verification commands (executed 2026-03-11):
+  - `go test ./internal/observability ./internal/agents/acpstdio -count=1`
+  - `go test ./cmd/ngent -count=1`
+  - `cd internal/webui/web && npm run build`
+  - `go test ./...`
+
+## Requirement 23: ACP Session Sidebar and Resume
+
+- Operation:
+  - create a thread/agent in the Web UI or API.
+  - query `GET /v1/threads/{threadId}/sessions` and verify the first page of ACP sessions plus `nextCursor`.
+  - request the next page through the returned cursor.
+  - start a turn on a thread without `sessionId` and observe session binding.
+  - start a follow-up turn on the now-bound thread.
+- Expected:
+  - the backend proxies ACP `session/list` through `GET /v1/threads/{threadId}/sessions`.
+  - response includes `supported`, `sessions`, and `nextCursor`.
+  - the Web UI renders a right-side session sidebar with:
+    - first-page load on active thread selection.
+    - `Show more` pagination when `nextCursor` is present.
+    - `New session` action that clears the selected `sessionId`.
+  - turn SSE emits `session_bound`, and the thread persists `agentOptions.sessionId`.
+  - once a thread is session-bound, subsequent prompt building no longer injects prior local turns into the provider prompt.
+- Verification commands (executed 2026-03-11):
+  - `go test ./internal/httpapi -run 'TestThreadSessionsListEndpoint|TestTurnSessionBoundPersistsSessionIDAndSkipsContextInjection' -count=1`
+  - `cd internal/webui/web && npm run build`
+  - `go test ./...`

docs/API.md

@@ -22,6 +22,12 @@ This document defines the current HTTP API contract.
   - `duration_ms`
   - `resp_bytes`
 - Structured log `time` is emitted as UTC `time.DateTime` with second precision.
+- When server starts with `--debug=true`, stderr also emits `acp.message` debug entries for ACP JSON-RPC traffic with:
+  - `component`
+  - `direction` (`inbound|outbound`)
+  - `rpcType` (`request|response|notification`)
+  - `method` when present
+  - sanitized `rpc` payload with sensitive fields redacted
 
 ## Unified Error Envelope