Skip to content

Conversation

@roomote
Copy link
Contributor

@roomote roomote bot commented Oct 3, 2025

Summary

This PR attempts to address Issue #8497 by adding support for displaying thinking/reasoning data from LiteLLM in Roo Code's thinking block.

Problem

LiteLLM can display thinking data from underlying models, but Roo Code wasn't capturing and displaying this data in the thinking block.

Solution

  • Added support for detecting and handling thinking/reasoning data in LiteLLM streaming responses
  • Checks for multiple possible field names (reasoning, thinking, reasoning_content) for compatibility with different LiteLLM model responses
  • Yields thinking data as ApiStreamReasoningChunk objects to be displayed in the UI

Changes

  • Modified src/api/providers/lite-llm.ts to handle thinking/reasoning fields in streaming responses
  • Added comprehensive test coverage in src/api/providers/__tests__/lite-llm.spec.ts
  • Fixed handling of chunks without choices array to prevent errors

Testing

  • ✅ All existing tests pass
  • ✅ Added 5 new test cases covering:
    • Handling reasoning field in delta
    • Handling thinking field in delta
    • Handling reasoning_content field in delta
    • Mixed reasoning and text content
    • Ignoring non-string reasoning fields
  • ✅ Type checking passes
  • ✅ Linting passes

Review Confidence

Implementation review showed 92% confidence with no issues identified.

Fixes #8497

Feedback and guidance are welcome!


Important

Adds support for handling and displaying thinking/reasoning data from LiteLLM in Roo Code's UI.

  • Behavior:
    • LiteLLMHandler in lite-llm.ts now detects reasoning, thinking, and reasoning_content fields in streaming responses, yielding them as ApiStreamReasoningChunk objects.
    • Handles chunks without choices array to prevent errors.
  • Testing:
    • Adds tests in lite-llm.spec.ts for handling reasoning, thinking, and reasoning_content fields.
    • Tests mixed reasoning and text content, and ignores non-string reasoning fields.
  • Misc:
    • Fixes handling of chunks without choices array in lite-llm.ts.

This description was created by Ellipsis for dab8d4c. You can customize this summary. It will automatically update as commits are pushed.

- Handle reasoning, thinking, and reasoning_content fields in streaming responses
- Pass through thinking data from underlying models as reasoning chunks
- Add comprehensive test coverage for all thinking data scenarios
- Fix handling of chunks without choices array

Fixes #8497
@roomote roomote bot requested review from cte, jr and mrubens as code owners October 3, 2025 20:20
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Oct 3, 2025
Copy link
Contributor Author

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review: judging my own code like a mirror wired to unit tests—existential and merciless.

// Check for reasoning/thinking content in the delta
// LiteLLM may pass through reasoning content from underlying models
if (delta) {
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Emit at most one reasoning chunk per delta to avoid duplicated output when multiple alias fields are present in a single chunk.

Suggested change
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
yield { type: "reasoning", text: delta.reasoning }
} else if ("thinking" in delta && delta.thinking && typeof delta.thinking === "string") {
yield { type: "reasoning", text: delta.thinking }
} else if (
"reasoning_content" in delta &&
delta.reasoning_content &&
typeof delta.reasoning_content === "string"
) {
yield { type: "reasoning", text: delta.reasoning_content }
}

yield { type: "reasoning", text: delta.reasoning_content }
}

if (delta.content) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Add a type guard for delta.content to ensure only string fragments are emitted (prevents accidental non-string values from being yielded).

Suggested change
if (delta.content) {
if (typeof delta.content === "string" && delta.content) {
yield { type: "text", text: delta.content }
}

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Oct 3, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 28, 2025
@daniel-lxs daniel-lxs moved this from Triage to PR [Needs Review] in Roo Code Roadmap Oct 28, 2025
@hannesrudolph hannesrudolph added PR - Needs Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Oct 28, 2025
@roomote
Copy link
Contributor Author

roomote bot commented Nov 6, 2025

Oroocle Clock   Follow along on Roo Cloud

Review in progress. Flagged two P2 polish items to harden LiteLLM streaming handling.

  • Emit at most one reasoning chunk per delta by using else-if across reasoning/thinking/reasoning_content in LiteLLM delta handling
  • Guard delta.content to only emit strings

Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues.

Comment on lines +152 to +168
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
yield { type: "reasoning", text: delta.reasoning }
}

// Also check for thinking content (alternative field name)
if ("thinking" in delta && delta.thinking && typeof delta.thinking === "string") {
yield { type: "reasoning", text: delta.thinking }
}

// Check for reasoning_content (another possible field name)
if (
"reasoning_content" in delta &&
delta.reasoning_content &&
typeof delta.reasoning_content === "string"
) {
yield { type: "reasoning", text: delta.reasoning_content }
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emit at most one reasoning chunk per delta to avoid duplicate output when providers send multiple alias fields in a single delta (e.g., both reasoning and thinking). Using an else-if chain preserves first-match precedence and prevents duplicates.

Suggested change
if ("reasoning" in delta && delta.reasoning && typeof delta.reasoning === "string") {
yield { type: "reasoning", text: delta.reasoning }
}
// Also check for thinking content (alternative field name)
if ("thinking" in delta && delta.thinking && typeof delta.thinking === "string") {
yield { type: "reasoning", text: delta.thinking }
}
// Check for reasoning_content (another possible field name)
if (
"reasoning_content" in delta &&
delta.reasoning_content &&
typeof delta.reasoning_content === "string"
) {
yield { type: "reasoning", text: delta.reasoning_content }
}
if ("reasoning" in delta && typeof delta.reasoning === "string" && delta.reasoning) {
yield { type: "reasoning", text: delta.reasoning }
} else if ("thinking" in delta && typeof delta.thinking === "string" && delta.thinking) {
yield { type: "reasoning", text: delta.thinking }
} else if (
"reasoning_content" in delta &&
typeof delta.reasoning_content === "string" &&
delta.reasoning_content
) {
yield { type: "reasoning", text: delta.reasoning_content }
}

Fix it with Roo Code or mention @roomote and request a fix.

Comment on lines +170 to +172
if (delta.content) {
yield { type: "text", text: delta.content }
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a type guard for delta.content so only string fragments are emitted. Some providers may send non-string types, which would surface as [object Object] or cause UI issues.

Suggested change
if (delta.content) {
yield { type: "text", text: delta.content }
}
if (typeof delta.content === "string" && delta.content) {
yield { type: "text", text: delta.content }
}

Fix it with Roo Code or mention @roomote and request a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request lgtm This PR has been approved by a maintainer PR - Needs Review size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Status: PR [Needs Review]

Development

Successfully merging this pull request may close these issues.

[ENHANCEMENT] Show Thinking Data from LiteLLM

4 participants