-
Notifications
You must be signed in to change notification settings - Fork 2.6k
fix: handle Gemini thinking-only responses to prevent empty assistant message error #6988
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
… message error - Add tracking for whether any actual content (not just reasoning) was yielded in Gemini handler - Yield empty text chunk if only reasoning content was provided to ensure assistantMessage is not empty - Improve error message in Task.ts to be more informative for Gemini-specific issues - Add comprehensive tests for thinking-only response scenarios Fixes #6986
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewing my own code is like debugging in a mirror - everything looks backwards but the bugs are still mine.
|
|
||
| // If we only got reasoning content and no actual text, yield an empty text chunk | ||
| // This ensures the assistant message won't be empty | ||
| if (!hasYieldedContent) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we add a comment here explaining why we yield an empty text chunk? Something like:
| if (!hasYieldedContent) { | |
| // If we only got reasoning content and no actual text, yield an empty text chunk | |
| // This ensures the assistant message won't be empty and prevents the | |
| // "language model did not provide any assistant messages" error | |
| if (!hasYieldedContent) { | |
| yield { type: "text", text: "" } | |
| } |
This would help future maintainers understand the reasoning behind this fix.
| @@ -0,0 +1,244 @@ | |||
| // npx vitest run src/api/providers/__tests__/gemini-thinking-only.spec.ts | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice comprehensive test coverage for the thinking-only scenarios! Consider whether these tests should be integrated into the main gemini.spec.ts file for better organization, or if keeping them separate is intentional for clarity. Both approaches have merit - separate files make the specific issue easier to find, while integration keeps all Gemini tests together.
| let errorMessage = "Unexpected API Response: The language model did not provide any assistant messages." | ||
|
|
||
| if (isGeminiModel) { | ||
| errorMessage += |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The enhanced error message is helpful! Could we make it even more specific by mentioning this often happens with tool function calls? Something like:
| errorMessage += | |
| if (isGeminiModel) { | |
| errorMessage += | |
| " This can occur with Gemini models when they are in 'thinking' mode but don't produce any actual response content, often during tool function calls. The model may need to be prompted again or the request may need to be retried." | |
| } else { |
|
Closing for now until we get more information about this issue |
Fixes #6986
Problem
When using Gemini 2.5 Pro, approximately every 2nd response was being interrupted with the error:
This occurred when Gemini was in "thinking" mode but didn't produce any actual response content, only reasoning/thinking content.
Solution
hasYieldedContentflag in the Gemini handler to track whether any actual text content (not just reasoning) was yieldedassistantMessageis not emptyTesting
Impact
This fix ensures that Gemini 2.5 Pro users won't experience interruptions when the model is in thinking mode, providing a smoother experience when using tool functions.
Important
Fixes Gemini 2.5 Pro thinking-only response issue by ensuring non-empty assistant messages and improving error handling.
hasYieldedContentflag inGeminiHandleringemini.tsto track if actual content is yielded.assistantMessage.Task.tsfor Gemini-specific issues.gemini-thinking-only.spec.tsto test thinking-only, mixed content, and empty stream scenarios.This description was created by
for f32a78d. You can customize this summary. It will automatically update as commits are pushed.