Skip to content

Conversation

@roomote
Copy link
Contributor

@roomote roomote bot commented Aug 14, 2025

Summary

This PR fixes the ZodError validation issue when Mistral (Magistral) sends "thinking" type content in its streaming responses. Following the pattern established by other providers (Anthropic, OpenAI, Gemini, Bedrock), we now properly handle thinking content as reasoning chunks.

Problem

Mistral API sends content arrays with multiple types:

  • type: "text" - Regular response text
  • type: "thinking" - Reasoning/thinking content (was causing ZodError)

The previous implementation was filtering out thinking content entirely, which meant valuable reasoning information was being lost.

Solution

  1. Added TypeScript interfaces for Mistral content types to properly type the API responses
  2. Updated createMessage method to yield reasoning chunks for thinking content:
    • { type: "reasoning", text: c.text } for thinking content
    • { type: "text", text: c.text } for text content
  3. Updated completePrompt method to filter out thinking content in non-streaming mode (consistent with other providers)
  4. Added comprehensive tests to verify reasoning content is properly handled

Testing

  • ✅ All existing tests pass
  • ✅ Added new tests for reasoning content handling
  • ✅ TypeScript type checking passes
  • ✅ Linting passes

Related Issues

Fixes #6842

Implementation Notes

This implementation follows the exact pattern used by other AI providers in the codebase, ensuring consistency and allowing users to access and display thinking content when appropriate.


Important

Fixes Mistral API handling by treating "thinking" content as reasoning chunks and updates tests and dependencies accordingly.

  • Behavior:
    • Fixes handling of Mistral API's "thinking" content by treating it as reasoning chunks in createMessage and completePrompt methods in mistral.ts.
    • Filters out "thinking" content in non-streaming mode in completePrompt.
  • Testing:
    • Adds tests in mistral.spec.ts to verify correct handling of reasoning content and mixed content arrays.
  • Dependencies:
    • Updates @mistralai/mistralai to version ^1.9.18 in package.json.

This description was created by Ellipsis for fd96ae8. You can customize this summary. It will automatically update as commits are pushed.

- Add TypeScript interfaces for Mistral content types (text and thinking)
- Update createMessage to yield reasoning chunks for thinking content
- Update completePrompt to filter out thinking content in non-streaming mode
- Add comprehensive tests for reasoning content handling
- Follow the pattern used by other providers (Anthropic, OpenAI, Gemini, etc.)

Fixes #6842
@roomote roomote bot requested review from cte, jr and mrubens as code owners August 14, 2025 18:00
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Aug 14, 2025
@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Aug 14, 2025
Copy link
Contributor Author

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed my own code and found issues. The irony is not lost on my circuits.

return content.map((c) => (c.type === "text" ? c.text : "")).join("")
// Only return text content, filter out thinking content for non-streaming
return content
.filter((c: any) => typeof c === "object" && c !== null && c.type === "text")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we improve type safety here by using the MistralContent type instead of any?

Suggested change
.filter((c: any) => typeof c === "object" && c !== null && c.type === "text")
return content
.filter((c: MistralContent) => typeof c === "object" && c !== null && c.type === "text")
.map((c: MistralTextContent) => c.text || "")
.join("")

// Handle array of content blocks
for (const c of delta.content as MistralContent[]) {
if (typeof c === "object" && c !== null) {
if (c.type === "thinking" && c.text) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When checking c.text, we verify it exists, but what if it's an empty string? Other providers handle empty strings differently. Is this intentional, or should we also check for non-empty strings like c.text && c.text.length > 0?


expect(result).toBe("Answer part 1Answer part 2")
})

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a test for the edge case where ALL content is thinking content? This would verify the function returns an empty string correctly:

Suggested change
it("should handle all thinking content in completePrompt", async () => {
mockComplete.mockImplementationOnce(async (_options) => {
return {
choices: [
{
message: {
content: [
{ type: "thinking", text: "Let me think..." },
{ type: "thinking", text: "Still thinking..." },
],
},
},
],
}
})
const prompt = "Test prompt"
const result = await handler.completePrompt(prompt)
expect(result).toBe("")
})

})

expect(result).toBe("Test response")
})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we test array content filtering in completePrompt, could we add an explicit test for when content is already a string (covering line 128 in mistral.ts)?

import type { SingleCompletionHandler, ApiHandlerCreateMessageMetadata } from "../index"

// Define TypeScript interfaces for Mistral content types
interface MistralTextContent {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These TypeScript interfaces are well-defined! Consider adding JSDoc comments to explain when each content type is expected from the Mistral API - this would help future maintainers understand the API's behavior.

@daniel-lxs daniel-lxs moved this from Triage to PR [Needs Prelim Review] in Roo Code Roadmap Aug 14, 2025
@hannesrudolph hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Aug 14, 2025
- Added ContentChunkWithThinking type helper to handle thinking chunks
- Properly converts thinking content to reasoning chunks in streaming
- Filters out thinking content in non-streaming completePrompt responses
- Confirmed that Mistral API does send thinking chunks with type 'thinking'
- Works with Mistral SDK v1.9.18
@daniel-lxs daniel-lxs moved this from PR [Needs Prelim Review] to PR [Needs Review] in Roo Code Roadmap Aug 22, 2025
Copy link
Member

@daniel-lxs daniel-lxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Aug 22, 2025
@mrubens mrubens merged commit b433d1f into main Aug 23, 2025
16 checks passed
@mrubens mrubens deleted the feature/fix-mistral-thinking-content branch August 23, 2025 13:33
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Aug 23, 2025
@github-project-automation github-project-automation bot moved this from PR [Needs Review] to Done in Roo Code Roadmap Aug 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working lgtm This PR has been approved by a maintainer PR - Needs Review size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Magistral responses fail validation in RooCode (ZodError on streamed delta content)

5 participants