Context
Pack SSE streams (/packs/{id}/run/stream and the legacy pipeline stream) currently emit phase-level events (phase_started, phase_completed, pipeline_completed). Clients see nothing between phase boundaries — no incremental output while the LLM generates.
Token-level streaming (each LLM token forwarded as it is generated) is the UX baseline users expect today, and a prerequisite for AG-UI support (#planned).
Scope
- Propagate LangGraph token streaming (
stream_mode="messages" or equivalent) through agent nodes up to the SSE endpoints, alongside the existing phase events (keep those — they are useful for progress UI).
- Decide the event schema: e.g.
{"type": "token", "phase": "analysis", "content": "..."}.
CostTracker interaction: token accounting on an in-flight stream, and what happens when the budget is exhausted mid-stream (emit an explicit error event, consistent with the existing SSE error behaviour).
- Structured-output packs (
StructuredLLMPack) don't stream naturally (validated JSON) — token streaming may apply only to free-text phases, with structured packs keeping phase-level events. To be settled at design time.
Non-goals
- No UI work; this is transport-level only.
Context
Pack SSE streams (
/packs/{id}/run/streamand the legacy pipeline stream) currently emit phase-level events (phase_started,phase_completed,pipeline_completed). Clients see nothing between phase boundaries — no incremental output while the LLM generates.Token-level streaming (each LLM token forwarded as it is generated) is the UX baseline users expect today, and a prerequisite for AG-UI support (#planned).
Scope
stream_mode="messages"or equivalent) through agent nodes up to the SSE endpoints, alongside the existing phase events (keep those — they are useful for progress UI).{"type": "token", "phase": "analysis", "content": "..."}.CostTrackerinteraction: token accounting on an in-flight stream, and what happens when the budget is exhausted mid-stream (emit an expliciterrorevent, consistent with the existing SSE error behaviour).StructuredLLMPack) don't stream naturally (validated JSON) — token streaming may apply only to free-text phases, with structured packs keeping phase-level events. To be settled at design time.Non-goals