Skip to content

Conversation

@Ruakij
Copy link
Contributor

@Ruakij Ruakij commented May 23, 2025

Related GitHub Issue

Closes: #3008

Description

This PR introduces manual control for the "thinking token limit" if supported. Not setting it will use default handling by provider/model e.g. Gemini will then adjust the budget according to the request as their docs.
Support for this feature is determined by checking if modelInfo has modelMaxThinkingTokens. (from a quick glance, this seems to correlate with the only providers for which we support it currently: gemini and vertex)

Key changes:

  • Added manualThinkingBudgetEnabled setting (defaults to false), which sets max thinking token limit to undefined when false, resulting it it not being added to the request.
  • Migrate settings for existing profiles with modelMaxThinkingTokens already set (sets manualThinkingBudgetEnabled to true).
  • Add Checkbox to enable manual control before the thinking tokens slider.
  • Add Tests for UI

Test Procedure

  1. Unit-Tests
  2. Manual:
    • In provider settings, select a model with thinking support.
    • Default (Automatic Budget): "Manually control" checkbox is unchecked. Slider is hidden.
    • Manual Override: Check "Manually control". Slider appears. Set budget.
    • Unsupported Providers/Models (e.g. OpenRouter with sonnet 3.7 thinking): Checkbox is disabled.
    • Migration: Existing configs with modelMaxThinkingTokens should have manual control enabled.

Type of Change

  • 🐛 Bug Fix: Non-breaking change that fixes an issue.
  • New Feature: Non-breaking change that adds functionality.
  • 💥 Breaking Change: Fix or feature that would cause existing functionality to not work as expected.
  • ♻️ Refactor: Code change that neither fixes a bug nor adds a feature.
  • 💅 Style: Changes that do not affect the meaning of the code (white-space, formatting, etc.).
  • 📚 Documentation: Updates to documentation files.
  • ⚙️ Build/CI: Changes to the build process or CI configuration.
  • 🧹 Chore: Other changes that don't modify src or test files.

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • [ x Self-Review: I have performed a thorough self-review of my code.
  • Code Quality:
    • My code adheres to the project's style guidelines.
    • There are no new linting errors or warnings (npm run lint).
    • All debug code (e.g., console.log) has been removed.
  • Testing:
    • New and/or updated tests have been added to cover my changes.
    • All tests pass locally (npm test).
    • The application builds successfully with my changes.
  • Branch Hygiene: My branch is up-to-date (rebased) with the main branch.
  • Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
  • Changeset: A changeset has been created using npm run changeset if this PR includes user-facing changes or dependency updates.
  • Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

Unchecked:
Screenshot_20250523_104902
Checked:
Screenshot_20250523_104914
Disabled on OpenRouter:
Screenshot_20250523_105128

Documentation Updates

  • Yes, documentation updates are required.
    • Explain new "Manually control thinking token limit" setting, default automatic behavior, and manual override. (if exists)

Additional Notes

Get in Touch

Discord: ruakij


Important

Introduces manual control for thinking token limits with UI updates, profile migration, and tests, affecting several files and settings.

  • Behavior:
    • Adds manualThinkingBudgetEnabled setting to control thinking token limits, defaulting to false.
    • Migrates existing profiles with modelMaxThinkingTokens to enable manual control.
    • Updates UI to include a checkbox for manual control and a slider for token limits.
  • Files and Functions:
    • Updates gemini.ts and vertex.ts to use setReasoningEffort for thinking token configuration.
    • Modifies ProviderSettingsManager.ts to include migration logic for manual thinking budget.
    • Adjusts model-params.ts and reasoning.ts to use shouldSetReasoningBudget.
  • Tests and Misc:
    • Adds tests for UI changes in ApiOptions.test.tsx and ThinkingBudget.tsx.
    • Renames enableReasoningEffort to setReasoningEffort across multiple files.
    • Updates translations in various settings.json files for new settings.

This description was created by Ellipsis for bcb4275. You can customize this summary. It will automatically update as commits are pushed.

@Ruakij
Copy link
Contributor Author

Ruakij commented May 23, 2025

-.- Rework of reasoning was just merged, guess i need to adjust this for the rework

@hannesrudolph hannesrudolph moved this from New to PR [Draft/WIP] in Roo Code Roadmap May 23, 2025
@hannesrudolph
Copy link
Collaborator

hannesrudolph commented May 23, 2025

@Ruakij sorry about that!

@hannesrudolph hannesrudolph moved this from PR [Draft / In Progress] to TEMP in Roo Code Roadmap May 26, 2025
@daniel-lxs daniel-lxs moved this from TEMP to PR [Draft / In Progress] in Roo Code Roadmap May 26, 2025
@Ruakij Ruakij force-pushed the feat/3008/Support-gemini-2.5-flash-determining-its-own-thinking-budget branch from 908960b to bcb4275 Compare May 30, 2025 06:45
@vercel
Copy link

vercel bot commented May 30, 2025

@Ruakij is attempting to deploy a commit to the Roo Code Team on Vercel.

A member of the Team first needs to authorize it.

@hannesrudolph hannesrudolph moved this from PR [Draft / In Progress] to PR [Needs Prelim Review] in Roo Code Roadmap Jun 2, 2025
@hannesrudolph hannesrudolph marked this pull request as ready for review June 2, 2025 20:03
@hannesrudolph hannesrudolph requested review from cte and mrubens as code owners June 2, 2025 20:03
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Jun 2, 2025
@hannesrudolph
Copy link
Collaborator

I moved this back into queue as it looked like @Ruakij made adjustments requested.

"useCustomBaseUrl": "Use custom base URL",
"useReasoning": "Enable reasoning",
"setReasoningEffort": "Set reasoning effort",
"implicitReasoning": "Provider will choose reasoning effort. Depending on the model this might automatic choose reasoning effort depending on the request.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There appears to be a typographical error in the sentence. Consider revising:

"Provider will choose reasoning effort. Depending on the model this might automatic choose reasoning effort depending on the request."

Suggested correction:

"Provider will choose reasoning effort. Depending on the model, this might automatically choose the reasoning effort based on the request."

Suggested change
"implicitReasoning": "Provider will choose reasoning effort. Depending on the model this might automatic choose reasoning effort depending on the request.",
"implicitReasoning": "Provider will choose reasoning effort. Depending on the model, this might automatically choose the reasoning effort based on the request.",

"getGlamaApiKey": "Glama APIキーを取得",
"useCustomBaseUrl": "カスタムベースURLを使用",
"useReasoning": "推論を有効化",
"setReasoningEffort": "推論の強度を設定",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typographical note: The term “推論の強度” in the setReasoningEffort entry is inconsistent with the subsequent keys, which use “推論努力”. Consider using consistent terminology for clarity.

Suggested change
"setReasoningEffort": "推論の強度を設定",
"setReasoningEffort": "推論努力を設定",

This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.

"getGlamaApiKey": "Lấy khóa API Glama",
"useCustomBaseUrl": "Sử dụng URL cơ sở tùy chỉnh",
"useReasoning": "Bật lý luận",
"setReasoningEffort": "Đặt nỗ lực lý luận",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: The term "lý luận" is used in the value for setReasoningEffort, but the following keys use "suy luận". It might be more consistent to change "Đặt nỗ lực lý luận" to "Đặt nỗ lực suy luận".

Suggested change
"setReasoningEffort": "Đặt nỗ lực luận",
"setReasoningEffort": "Đặt nỗ lực suy luận",

This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.

@Ruakij
Copy link
Contributor Author

Ruakij commented Jun 2, 2025

I had switched the existing Set reasoning button to instead set thinking budget to unset when unchecked.
But this is not consistent across providers and ideally we could get rid of the thinking and non-thinking models in the list in the future and just have a button (i think thats what the button is/was for)

So this needs some more work, dont think i can finish this at the moment.

@Ruakij Ruakij marked this pull request as draft June 2, 2025 20:23
@hannesrudolph
Copy link
Collaborator

I had switched the existing Set reasoning button to instead set thinking budget to unset when unchecked. But this is not consistent across providers and ideally we could get rid of the thinking and non-thinking models in the list in the future and just have a button (i think thats what the button is/was for)

So this needs some more work, dont think i can finish this at the moment.

Ok will close this and allow someone else to take up the issue.

@github-project-automation github-project-automation bot moved this from PR [Needs Prelim Review] to Done in Roo Code Roadmap Jun 2, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Jun 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request PR - Draft / In Progress size:L This PR changes 100-499 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Support gemini-2.5-flash determining its own thinking budget

3 participants