Skip to content

Conversation

@cte
Copy link
Collaborator

@cte cte commented May 24, 2025

Description

OpenRouter only tells you if a model supports the reasoning parameter in the API; it doesn't tell you if the model honors a token budget for thinking. As such we need to continue to maintain a hardcoded list.


Important

Fixes reasoning budget handling for Gemini 2.5 Flash on OpenRouter by updating model constants and processing logic.

  • Behavior:
    • Fixes reasoning budget handling for google/gemini-2.5-flash-preview-05-20 in openrouter.ts and litellm.ts.
    • Updates parseOpenRouterModel() to correctly set maxTokens and reasoning budget flags.
  • Constants:
    • Renames PROMPT_CACHING_MODELS to OPEN_ROUTER_PROMPT_CACHING_MODELS and COMPUTER_USE_MODELS to OPEN_ROUTER_COMPUTER_USE_MODELS.
    • Adds OPEN_ROUTER_REASONING_BUDGET_MODELS and OPEN_ROUTER_REQUIRED_REASONING_BUDGET_MODELS to api.ts.
  • Tests:
    • Updates tests in litellm.test.ts and openrouter.spec.ts to reflect new model constants and reasoning budget handling.

This description was created by Ellipsis for a7cba79. You can customize this summary. It will automatically update as commits are pushed.

import axios from "axios"
import { getLiteLLMModels } from "../litellm"
import { COMPUTER_USE_MODELS } from "../../../../shared/api"
import { OPEN_ROUTER_COMPUTER_USE_MODELS } from "../../../../shared/api"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed this to make it clear that it's OpenRouter specific, so it's a bit weird that we're using it in LiteLLM.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... I think the idea is to pull this from LiteLLM config instead soon, right @slytechnical?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. The addition of support for that has been merged in on LiteLLM's side , but I figured we should wait till that goes live in their next release before switching to it on our end

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LiteLLM said that we didn’t need to wait for a release since it pulls from hosted config - probably worth confirming though.

modelInfo={selectedModelInfo}
/>

<pre>{JSON.stringify(selectedModelInfo, null, 2)}</pre>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a <pre> block rendering JSON.stringify(selectedModelInfo, null, 2). Consider removing or conditionally rendering it (e.g. only in debug mode) to avoid exposing internal details in production.

Suggested change
<pre>{JSON.stringify(selectedModelInfo, null, 2)}</pre>

modelInfo={selectedModelInfo}
/>

<pre>{JSON.stringify(selectedModelInfo, null, 2)}</pre>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intentional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No 🙈

@mrubens mrubens merged commit 77a5b6c into main May 24, 2025
10 checks passed
@mrubens mrubens deleted the cte/fix-open-router-gemini-reasoning-budget branch May 24, 2025 23:46
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap May 24, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap May 24, 2025
SmartManoj pushed a commit to SmartManoj/Raa-Code that referenced this pull request Jun 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants