feat/cancel-generation - stop an ongoing generation query

[udaykumar-dhokia]

This PR introduces the ability for users to cancel/stop an ongoing LLM generation mid-stream. Previously, queries would continue running in the background even if the user deleted the chat or navigated away, wasting local or cloud compute resources. (Resolves #989)

Changes Made

Backend

• Session Manager: Integrated AbortController to allow consumers to listen for cancellation via session.signal.
• API Route (/api/chat): Upgraded the disconnect listener to call session.abort(), terminating the entire background agent process.
• LLM Providers: Updated OpenAI LLM (and derived classes) and OllamaLLM to natively accept and respect an AbortSignal in their streaming SDK calls, breaking the stream loop cleanly.
• Agent Layer: Added abort checks to the multi-iteration Researcher loop and SearchAgent string generation, ensuring partial block data is saved to the database on cancellation.

Frontend

• Chat Hook (useChat): Added a cancelMessage function exposed via context, managing a local fetch AbortController ref.
• Message Input UI: Replaced the "Send" arrow with a red "Stop" (Square) button while the stream is loading, which triggers cancelMessage.

Testing Instructions

1. Start a long-running query (e.g., "Write a very long essay...").
2. While the UI says "Brainstorming..." or is actively streaming text, click the red Stop button.
3. Verify that the streaming stops immediately.
4. Verify that the Stop button gracefully returns to the Send button.
5. Check the backend logs/terminal to confirm no further tokens are generated after cancellation.

---

Summary by cubic

Adds a stop button to cancel an in‑progress LLM generation. Streams now abort cleanly across the stack and save partial output to avoid wasted compute. Resolves #989.

• New Features
  • Frontend: Show a red Stop button while streaming; added useChat.cancelMessage to abort fetch and reset UI state.
  • Backend: SessionManager now uses AbortController; /api/chat aborts the session on disconnect; OpenAI/Ollama providers and agents accept AbortSignal to halt streaming loops and persist partial blocks while suppressing abort errors.

^{Written for commit ff24730. Summary will update on new commits.}

[cubic-dev-ai]

3 issues found across 12 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/lib/hooks/useChat.tsx">

<violation number="1" location="src/lib/hooks/useChat.tsx:749">
P2: Cancellation is only wired for `sendMessage`; reconnect streaming fetch is missing `signal`, so Stop cannot abort reconnected generations.</violation>

<violation number="2" location="src/lib/hooks/useChat.tsx:833">
P1: Cancelling generation leaves `messages` and `chatHistory` out of sync, which breaks `rewrite` index-based history slicing and can send wrong conversation context.</violation>
</file>

<file name="src/lib/agents/search/index.ts">

<violation number="1" location="src/lib/agents/search/index.ts:97">
P2: Abort handling is checked too late: `searchAsync` still waits for non-cancelable widget execution to finish before returning on cancel.</violation>
</file>

---

Since this is your first cubic review, here's how it works:

• cubic automatically reviews your code and comments on bugs and improvements
• Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
• Add one-off context when rerunning by tagging @cubic-dev-ai with guidance or docs links (including llms.txt)
• Ask questions if you need clarification on any suggestion

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P1: Cancelling generation leaves messages and chatHistory out of sync, which breaks rewrite index-based history slicing and can send wrong conversation context.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/lib/hooks/useChat.tsx, line 833:

<comment>Cancelling generation leaves `messages` and `chatHistory` out of sync, which breaks `rewrite` index-based history slicing and can send wrong conversation context.</comment>

<file context>
@@ -742,67 +745,97 @@ export const ChatProvider = ({ children }: { children: React.ReactNode }) => {
+    setMessages((prev) =>
+      prev.map((msg) =>
+        msg.status === 'answering'
+          ? { ...msg, status: 'completed' as const }
+          : msg,
+      ),
</file context>

[cubic-dev-ai]

P2: Cancellation is only wired for sendMessage; reconnect streaming fetch is missing signal, so Stop cannot abort reconnected generations.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/lib/hooks/useChat.tsx, line 749:

<comment>Cancellation is only wired for `sendMessage`; reconnect streaming fetch is missing `signal`, so Stop cannot abort reconnected generations.</comment>

<file context>
@@ -742,67 +745,97 @@ export const ChatProvider = ({ children }: { children: React.ReactNode }) => {
-          key: embeddingModelProvider.key,
-          providerId: embeddingModelProvider.providerId,
+    const controller = new AbortController();
+    abortControllerRef.current = controller;
+
+    try {
</file context>

[cubic-dev-ai]

P2: Abort handling is checked too late: searchAsync still waits for non-cancelable widget execution to finish before returning on cancel.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/lib/agents/search/index.ts, line 97:

<comment>Abort handling is checked too late: `searchAsync` still waits for non-cancelable widget execution to finish before returning on cancel.</comment>

<file context>
@@ -94,6 +94,23 @@ class SearchAgent {
       searchPromise,
     ]);
 
+    if (session.signal.aborted) {
+      await db
+        .update(messages)
</file context>