Skip to content

feat(observer): establish server-side observability primitives for TUI#3

Merged
andrewmelchor merged 3 commits intomainfrom
feat/tui-phase-1-bootstrap
Jan 21, 2026
Merged

feat(observer): establish server-side observability primitives for TUI#3
andrewmelchor merged 3 commits intomainfrom
feat/tui-phase-1-bootstrap

Conversation

@andrewmelchor
Copy link
Copy Markdown
Member

@andrewmelchor andrewmelchor commented Jan 21, 2026

Summary

This introduces flow/execution correlation and server-backed storage so a TUI can observe runs in real-time (SSE) and fetch full execution details on demand.

  • flows: create/finish + TTL/limits + flow-scoped sequencing
  • executions: reqExecId, requestQueued, lifecycle tracking, error finalization
  • storage: GET execution detail by flowId/reqExecId
  • workspace: list .http files and requests for navigation
  • security: session variable redaction; SSE filter requirements under auth
  • core: createRemoteClient for “single import change” observability + ordered var sync
  • tests: update parse calls for includeDiagnostics typing; Biome formatting

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring
  • Chore (dependency update, build script changes, etc.)

How Has This Been Tested?

  • Unit tests
  • E2E tests
  • Manual testing (please describe)

Checklist

  • I have followed the contributing guidelines
  • I have performed a self-review of my own code
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the documentation accordingly
  • My changes generate no new warnings

This introduces flow/execution correlation and server-backed storage so a TUI
can observe runs in real-time (SSE) and fetch full execution details on demand.

- flows: create/finish + TTL/limits + flow-scoped sequencing
- executions: reqExecId, requestQueued, lifecycle tracking, error finalization
- storage: GET execution detail by flowId/reqExecId
- workspace: list .http files and requests for navigation
- security: session variable redaction; SSE filter requirements under auth
- core: createRemoteClient for “single import change” observability + ordered var sync
- tests: update parse calls for includeDiagnostics typing; Biome formatting
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages bot commented Jan 21, 2026

Deploying docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 82373d2
Status: ✅  Deploy successful!
Preview URL: https://ea0ba4b3.docs-6vq.pages.dev
Branch Preview URL: https://feat-tui-phase-1-bootstrap.docs-6vq.pages.dev

View logs

@andrewmelchor
Copy link
Copy Markdown
Member Author

@greptile

@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Jan 21, 2026

Greptile Summary

Introduces comprehensive observability infrastructure for the TUI by establishing flow-based execution correlation and server-backed storage. The implementation enables real-time monitoring through SSE with flow-scoped event sequencing.

Key Changes:

  • Remote Client (packages/core/src/remote-client.ts): New createRemoteClient with promise-chained variable synchronization to prevent race conditions during rapid setVariable calls
  • Flow Management (packages/app/src/server/service.ts): Implements flow lifecycle (create/finish) with TTL-based cleanup, execution storage with reqExecId correlation, and flow-scoped sequence numbering
  • Security (packages/app/src/server/service.ts:255-278): Deep variable sanitization with circular reference detection using WeakSet; redacts sensitive keys (token, password, apiKey, etc.) in nested objects and arrays
  • Event Streaming (packages/app/src/server/app.ts:235-306, packages/app/src/server/events.ts): SSE endpoint with session/flow filtering; requires sessionId or flowId when auth enabled to prevent cross-session leakage
  • Workspace Discovery: New endpoints to list .http files and requests for TUI navigation
  • Execution Lifecycle: Tracks status transitions (pending → running → success/failed) with error finalization for Observer Mode

Issues Found:

  • SSE auth filter doesn't validate sessionId/flowId existence (line 238 in app.ts)
  • Flow eviction may remove active unfinished flows (line 393 in service.ts)
  • Execution status logic may incorrectly mark pending executions as successful (line 1158 in service.ts)

Confidence Score: 3/5

  • This PR has logical issues in auth filtering and execution status tracking that should be addressed before merge
  • Score reflects three logic issues: (1) SSE auth filter accepts arbitrary sessionId/flowId without validation enabling potential enumeration, (2) flow eviction doesn't respect finished state risking active flow removal, (3) execution status determination may incorrectly mark pending as successful. The observability architecture is sound with good security patterns (deep sanitization, circular ref detection), but the identified issues affect correctness and security.
  • Pay close attention to packages/app/src/server/app.ts (SSE auth validation) and packages/app/src/server/service.ts (flow eviction and execution status logic)

Important Files Changed

Filename Overview
packages/core/src/remote-client.ts New remote client with promise-chained variable sync; race condition warnings noted in previous thread
packages/app/src/server/service.ts Added flow/execution tracking, deep variable sanitization with circular detection, flow-scoped sequencing
packages/app/src/server/events.ts Event manager with flow/session filtering; probabilistic cleanup for run sequences
packages/app/src/server/app.ts Added flow endpoints and SSE filtering; requires sessionId/flowId when auth enabled
packages/app/test/server/service.test.ts Added comprehensive tests for deep sanitization with arrays/nested objects; missing circular ref test

Sequence Diagram

sequenceDiagram
    participant Client as Remote Client
    participant Server as treq serve
    participant EventMgr as Event Manager
    participant TUI as TUI Observer

    Note over Client,TUI: Initialization Phase
    Client->>Server: POST /session (variables)
    Server-->>Client: sessionId
    Client->>Server: POST /flows (sessionId, label)
    Server->>EventMgr: emit(flowStarted)
    EventMgr->>TUI: SSE: flowStarted
    Server-->>Client: flowId

    Note over Client,TUI: Execution Phase
    Client->>Client: setVariable(key, value)
    Client->>Server: PUT /session/{id}/variables
    Server-->>Client: 200 OK
    
    Client->>Server: POST /execute (flowId, sessionId, path)
    Server->>EventMgr: emit(requestQueued, seq=1)
    EventMgr->>TUI: SSE: requestQueued
    Server->>Server: Create pending execution
    Server->>Server: Parse & resolve config
    Server->>EventMgr: emit(fetchStarted, seq=2)
    EventMgr->>TUI: SSE: fetchStarted
    Server->>Server: Update exec status=running
    Server->>Server: HTTP request execution
    Server->>EventMgr: emit(fetchFinished, seq=3)
    EventMgr->>TUI: SSE: fetchFinished
    Server->>Server: Store execution (status=success)
    Server-->>Client: ExecuteResponse (reqExecId)

    Note over Client,TUI: Completion Phase
    Client->>Server: POST /flows/{flowId}/finish
    Server->>Server: Calculate flow summary
    Server->>EventMgr: emit(flowFinished, summary)
    EventMgr->>TUI: SSE: flowFinished
    Server-->>Client: FlowSummary

    Note over Client,TUI: Observer Retrieval
    TUI->>Server: GET /flows/{flowId}/executions/{reqExecId}
    Server-->>TUI: ExecutionDetail (sanitized headers/variables)
Loading

Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Comments (1)

  1. packages/app/src/server/service.ts, line 255-267 (link)

    logic: Arrays containing objects with sensitive keys won't be sanitized

    The function recursively sanitizes nested objects but skips arrays. If variables contains [{token: "secret"}], the sensitive data will leak.

16 files reviewed, 3 comments

Edit Code Review Agent Settings | Greptile

…sync

Sanitize nested variable structures including arrays to prevent sensitive-key leaks.
Queue remote client variable updates to avoid races and ensure executions see latest values.
Add regression tests for nested/array sanitization.
@andrewmelchor andrewmelchor marked this pull request as ready for review January 21, 2026 07:25
Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

16 files reviewed, 3 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +238 to 243
// Require sessionId or flowId when auth is enabled (prevents cross-session leakage)
if (config.token && !sessionId && !flowId) {
throw new ValidationError(
'sessionId query parameter is required when authentication is enabled'
'sessionId or flowId query parameter is required when authentication is enabled'
);
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Consider stronger enforcement when auth is enabled

When config.token is set, the code requires sessionId OR flowId, but doesn't validate that provided values exist. An attacker could subscribe to arbitrary sessionId/flowId values without verification.

Suggested change
// Require sessionId or flowId when auth is enabled (prevents cross-session leakage)
if (config.token && !sessionId && !flowId) {
throw new ValidationError(
'sessionId query parameter is required when authentication is enabled'
'sessionId or flowId query parameter is required when authentication is enabled'
);
}
// Require sessionId or flowId when auth is enabled (prevents cross-session leakage)
if (config.token && !sessionId && !flowId) {
throw new ValidationError(
'sessionId or flowId query parameter is required when authentication is enabled'
);
}
// Verify sessionId/flowId exist if provided (prevent enumeration)
if (sessionId && !service.getSessions().has(sessionId)) {
throw new SessionNotFoundError(sessionId);
}
if (flowId && !service.getFlows().has(flowId)) {
throw new FlowNotFoundError(flowId);
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/app/src/server/app.ts
Line: 238:243

Comment:
**logic:** Consider stronger enforcement when auth is enabled

When `config.token` is set, the code requires `sessionId` OR `flowId`, but doesn't validate that provided values exist. An attacker could subscribe to arbitrary sessionId/flowId values without verification.

```suggestion
    // Require sessionId or flowId when auth is enabled (prevents cross-session leakage)
    if (config.token && !sessionId && !flowId) {
      throw new ValidationError(
        'sessionId or flowId query parameter is required when authentication is enabled'
      );
    }
    
    // Verify sessionId/flowId exist if provided (prevent enumeration)
    if (sessionId && !service.getSessions().has(sessionId)) {
      throw new SessionNotFoundError(sessionId);
    }
    if (flowId && !service.getFlows().has(flowId)) {
      throw new FlowNotFoundError(flowId);
    }
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +393 to +404
// Evict oldest flow when limit reached
function evictOldestFlow(): void {
let oldest: Flow | null = null;
for (const flow of flows.values()) {
if (!oldest || flow.lastActivityAt < oldest.lastActivityAt) {
oldest = flow;
}
}
if (oldest) {
flows.delete(oldest.id);
}
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Flow eviction doesn't check if flow is finished

evictOldestFlow evicts based on lastActivityAt without checking flow.finished. An active, unfinished flow could be evicted while a client is still using it, causing subsequent operations to fail with FlowNotFoundError.

Consider either:

  • Skip evicting unfinished flows
  • Emit a warning when evicting an unfinished flow
  • Document that clients must call finishFlow before flows can be safely evicted
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/app/src/server/service.ts
Line: 393:404

Comment:
**logic:** Flow eviction doesn't check if flow is finished

`evictOldestFlow` evicts based on `lastActivityAt` without checking `flow.finished`. An active, unfinished flow could be evicted while a client is still using it, causing subsequent operations to fail with `FlowNotFoundError`.

Consider either:
- Skip evicting unfinished flows
- Emit a warning when evicting an unfinished flow
- Document that clients must call `finishFlow` before flows can be safely evicted

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +1158 to +1161
// Set final status: use tracker status if failed, otherwise success
exec.status = execTracker.status === 'failed' ? 'failed' : 'success';
exec.error = execTracker.error;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Status determination logic may be incorrect

Line 1159 sets exec.status = 'success' if execTracker.status !== 'failed', but execTracker.status could still be 'pending' or 'running' if fetchStarted event didn't fire. This would incorrectly mark executions as successful.

Suggested change
// Set final status: use tracker status if failed, otherwise success
exec.status = execTracker.status === 'failed' ? 'failed' : 'success';
exec.error = execTracker.error;
// Set final status: use tracker status if failed, otherwise success
exec.status = execTracker.status === 'failed' ? 'failed' :
execTracker.status === 'running' ? 'success' :
'success';
exec.error = execTracker.error;

Does the engine always emit fetchStarted before completion, or can executions succeed without transitioning through running state?

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/app/src/server/service.ts
Line: 1158:1161

Comment:
**logic:** Status determination logic may be incorrect

Line 1159 sets `exec.status = 'success'` if `execTracker.status !== 'failed'`, but `execTracker.status` could still be `'pending'` or `'running'` if `fetchStarted` event didn't fire. This would incorrectly mark executions as successful.

```suggestion
        // Set final status: use tracker status if failed, otherwise success
        exec.status = execTracker.status === 'failed' ? 'failed' : 
                      execTracker.status === 'running' ? 'success' : 
                      'success';
        exec.error = execTracker.error;
```

 Does the engine always emit fetchStarted before completion, or can executions succeed without transitioning through running state?

How can I resolve this? If you propose a fix, please make it concise.

Only evict finished flows when MAX_FLOWS is reached; if none are finished, throw FLOW_LIMIT_REACHED instead of deleting an in-progress flow. This prevents active clients from hitting FlowNotFoundError mid-flow.
@andrewmelchor andrewmelchor merged commit 1eff323 into main Jan 21, 2026
2 checks passed
@andrewmelchor andrewmelchor deleted the feat/tui-phase-1-bootstrap branch January 21, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant