-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Fix reasoning budget for Gemini 2.5 Flash on OpenRouter #3945
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| import axios from "axios" | ||
| import { getLiteLLMModels } from "../litellm" | ||
| import { COMPUTER_USE_MODELS } from "../../../../shared/api" | ||
| import { OPEN_ROUTER_COMPUTER_USE_MODELS } from "../../../../shared/api" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I renamed this to make it clear that it's OpenRouter specific, so it's a bit weird that we're using it in LiteLLM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah... I think the idea is to pull this from LiteLLM config instead soon, right @slytechnical?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct. The addition of support for that has been merged in on LiteLLM's side , but I figured we should wait till that goes live in their next release before switching to it on our end
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LiteLLM said that we didn’t need to wait for a release since it pulls from hosted config - probably worth confirming though.
| modelInfo={selectedModelInfo} | ||
| /> | ||
|
|
||
| <pre>{JSON.stringify(selectedModelInfo, null, 2)}</pre> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a <pre> block rendering JSON.stringify(selectedModelInfo, null, 2). Consider removing or conditionally rendering it (e.g. only in debug mode) to avoid exposing internal details in production.
| <pre>{JSON.stringify(selectedModelInfo, null, 2)}</pre> |
| modelInfo={selectedModelInfo} | ||
| /> | ||
|
|
||
| <pre>{JSON.stringify(selectedModelInfo, null, 2)}</pre> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this intentional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No 🙈
* request type * changeset
Description
OpenRouter only tells you if a model supports the
reasoningparameter in the API; it doesn't tell you if the model honors a token budget for thinking. As such we need to continue to maintain a hardcoded list.Important
Fixes reasoning budget handling for Gemini 2.5 Flash on OpenRouter by updating model constants and processing logic.
google/gemini-2.5-flash-preview-05-20inopenrouter.tsandlitellm.ts.parseOpenRouterModel()to correctly setmaxTokensand reasoning budget flags.PROMPT_CACHING_MODELStoOPEN_ROUTER_PROMPT_CACHING_MODELSandCOMPUTER_USE_MODELStoOPEN_ROUTER_COMPUTER_USE_MODELS.OPEN_ROUTER_REASONING_BUDGET_MODELSandOPEN_ROUTER_REQUIRED_REASONING_BUDGET_MODELStoapi.ts.litellm.test.tsandopenrouter.spec.tsto reflect new model constants and reasoning budget handling.This description was created by
for a7cba79. You can customize this summary. It will automatically update as commits are pushed.