Skip to content

feat: LLM streaming with UX improvements #29

Open
Jurij89 wants to merge 4 commits intomainfrom
feat/streaming
Open

feat: LLM streaming with UX improvements #29
Jurij89 wants to merge 4 commits intomainfrom
feat/streaming

Conversation

@Jurij89
Copy link
Contributor

@Jurij89 Jurij89 commented Feb 8, 2026

Summary

  • SSE streaming
    Server-sent events transport for real-time LLM responses on web (native path unchanged).
    Includes server-side processStreamingCompletion with .stream().invoke() fallback, client-side SSE parser, and rAF-throttled UI updates.

  • Plain text during streaming
    Renders raw <Text> instead of Markdown while tokens arrive, eliminating visual glitches from incomplete markup (unclosed fences, bold, partial links).
    Switches to full Markdown on completion.

  • Code block copy button
    Fenced code blocks display a copy icon (top-right) that swaps to ✓ Copied feedback for 2 seconds.

  • Scroll UX
    No auto-scroll during streaming or on completion (matches Claude/ChatGPT behavior).
    Scrolls to bottom only when the user sends a new message.

Files changed

File What
apps/agent/src/shared/chat.ts SSE types, processStreamingCompletion (server), makeStreamingCompletionRequest (client), writeSSE helper
apps/agent/src/server/index.ts Streaming middleware plugin (intercepts Accept: text/event-stream on /llm)
apps/agent/src/app/(protected)/chat.tsx streamingContent state, streaming functions, plain text rendering, scroll-on-send
apps/agent/src/components/Markdown.tsx CopyCodeButton component, custom fence render rule

Test plan

  • Send a message on web — verify tokens stream in real-time with plain text
  • Verify final message renders with full Markdown (bold, code blocks, lists)
  • Verify no auto-scroll during streaming — user can read at their own pace
  • Verify sending a new message scrolls to bottom
  • Verify code block copy button copies content and shows ✓ Copied feedback
  • Verify native (non-streaming) path is unaffected
  • Verify tool calls work correctly through the streaming path

Jurij Skornik added 2 commits February 8, 2026 12:56
…provements

  - Render plain <Text> during streaming instead of Markdown to eliminate
    visual glitches from incomplete markup (unclosed fences, bold, etc.)
  - Add copy button with "✓ Copied" feedback to fenced code blocks
  - Remove auto-scroll during streaming and on completion so users can
    read at their own pace (matches Claude/ChatGPT behavior)
  - Scroll to bottom only when user sends a new message
@Bojan131
Copy link
Contributor

Bojan131 commented Feb 9, 2026

Tested and some markdown parts seem not to be working. Bolded text and code blocks dont seem to be visible. When you ask it to put it in a code block it does do it but it doesn't put it automatically and also it has a delay where it puts it in a code block when the whole message stops streaming instead of streaming the code into the code block
Markdown
Please take a look at the image attached.
Other things work as expected

Bojan131 and others added 2 commits February 9, 2026 10:32
Replace the plain <Text> streaming renderer with the existing
<Markdown> component so formatting (bold, code blocks, lists, etc.)
appears progressively as tokens arrive, eliminating the visual "pop"
when the stream completes.

Add normalizeStreamingMarkdown() to close unclosed code fences
mid-stream, and stripThinkTags() to hide <think> blocks during
streaming. Remove now-dead streamingTextStyle.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants