Skip to content

Ask model to try again if it produced a response without text or tool call parts (e.g. only thinking) #2469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

Ask model to try again if it produced a response without text or tool call parts (e.g. only thinking) #2469

wants to merge 1 commit into from

Conversation

ethanabrooks
Copy link
Contributor

@ethanabrooks ethanabrooks commented Aug 7, 2025 •
edited by DouweM
Loading

Fixes #2480

@DouweM
Copy link
Collaborator

DouweM commented Aug 7, 2025

@ethanabrooks Thanks, which models have you seen output only thinking with no text or tool calls to follow it?

@DouweM DouweM self-assigned this Aug 7, 2025
@ethanabrooks ethanabrooks force-pushed the eb/handle-thinking-only-requests branch 7 times, most recently from e40e7a5 to a06e745 Compare August 7, 2025 20:52
DouweM
DouweM requested changes Aug 12, 2025
View reviewed changes
Copy link
Collaborator

@DouweM DouweM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ethanabrooks Thanks Ethan, this fix makes sense. Can you please add a test with a simulated thinking-only response using FunctionModel?

@DouweM DouweM changed the title add handling for thinking-only requests (currently causes UnexpectedModelBehavior) Ask model to try again if it produced a response without text or tool call parts (e.g. only thinking) Aug 12, 2025
@ethanabrooks ethanabrooks force-pushed the eb/handle-thinking-only-requests branch 11 times, most recently from de8ae11 to ee38ed0 Compare August 14, 2025 14:31
@ethanabrooks
Copy link
Contributor Author

ethanabrooks commented Aug 14, 2025 •
edited
Loading

@DouweM apologies for the premature PR submission -- actually didn't intend to submit until I heard back on the issue (#2480). Hopefully this revised version is a bit cleaner.

I put the logic for handling function-calls that have no text parts in another PR: #2551

Thanks.

@ethanabrooks ethanabrooks requested a review from DouweM August 14, 2025 14:39
@ethanabrooks ethanabrooks force-pushed the eb/handle-thinking-only-requests branch from ee38ed0 to f003b74 Compare August 14, 2025 14:42
@ethanabrooks
Copy link
Contributor Author

@ethanabrooks Thanks, which models have you seen output only thinking with no text or tool calls to follow it?

We have seen this with Gemini 2.5 pro.

@DouweM
Copy link
Collaborator

DouweM commented Aug 14, 2025

@ethanabrooks I made a couple of small tweaks locally, can you please enable the ability for maintainers to push to this branch? :)

@ethanabrooks
Copy link
Contributor Author

@ethanabrooks I made a couple of small tweaks locally, can you please enable the ability for maintainers to push to this branch? :)

@DouweM I think the opportunity to do this was when I created the PR... unfortunately I may have to close and reopen.

@ethanabrooks ethanabrooks reopened this Aug 14, 2025
@ethanabrooks
Copy link
Contributor Author

Issue must have been that the fork was behind a private org. This should work: #2556

@amiyapatanaik amiyapatanaik mentioned this pull request Aug 19, 2025
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DouweM DouweM Awaiting requested review from DouweM

Assignees

@DouweM DouweM

Labels
awaiting author revision
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

UnexpectedModelBehavior with Google Gemini thinking mode: "Received empty model response"
2 participants
@ethanabrooks @DouweM