Finer-grained control of Gemini models and Enhancement of Gemini Integration in Roo Code

[HahaBill]

Detailed Feature Proposal

What specific problem does this solve?

The main idea of this proposal is to provide users finer-grained control over Gemini and enhancing Gemini support in Roo Code.

However, setting context limit could extend for other API providers as well, this is currently an ongoing discussion and may not be included in the end.

This proposal aims to increase user control and experience with using Gemini as an API provider. There has been many documented issues:

Motivation

There has been attempts in the past with no sign of continuation and some requests are still pending:

Missing Integrations and Parameters

• Enable URL context + Grounding research: Gemini URL Context Support #4274
• Gemini embedding models: Codebase Indexing: Add Google Gemini as provider #3967
• Max Output Tokens: Decouple Max Tokens from Max Thinking Tokens #3355
• topP and topK - https://ai.google.dev/gemini-api/docs/prompting-strategies

Token-related issues

• Cannot strategically decrease context limits (input tokens): Add a custom context/token limit for models. #2422, Max Tokens config does not appear for gemini-2.5-pro; it should #3009, Allow setting a custom max context window for Google Gemini API provider (and/or universal max context window) #3717

Gemini User experience

• Token count decreasing Gemini experience: Gemini/Anthropic: Stop using remote token count APIs #3666
• Thinking cannot be turned off in new Gemini models, which leads to confusion: Add option for Gemini 0605 to enable reasoning but not set manual max-output-tokens and max-thinking-tokens #4407

Impact

Users will have a smoother and better experience using Gemini as their API provider, addressing user feedback and satisfying requests from links above. Furthermore, users will be able to manage cost more effectively by themselves.

In the future, there could another proposal to implement adaptive model selection and cost optimization with Gemini which builds on top of the proposal.

Structure

This proposal/issue will be partitioned into two GitHub issues:

1. Finer-grained control of Gemini models
2. Enhancement of Gemini Integration and UX in Roo Code

Note

This will be reviewed by Google Deepmind and further adjustment could be made. Furthermore, there are some duplicates but they seem to be inactive - those have to be verified.

How should this be solved?

In the current Roo Code's system, users can only modify temperatures of the Gemini models. Here is the list of missing configurable parameters, tools and integrations:

Gemini models

• topP and topK - add more parameters
• maxOutputTokens -> if no progress is made
• URL context - adding to tools
• Grounding Research - adding to tools

Token control

• contextLimit / maxContextWindow - still an ongoing disccusion whether to include, may not be only Gemini-specific

Gemini integrations

• Gemini embedding models of Codebase Indexing -> if no progress is made

Increase Experience with Gemini

• Removing token counting via APIs - using local token counting
• fix to rename the 'Enable reasoning' checkbox to a better option and function of 'Set Reasoning/Thinking Budget' for new Gemini models. Or come up with better UX to decrease confusion.

How will we know it works? (Acceptance Criteria)

Finer-grained control of Gemini models

Case 1: Changing parameters and configurations

1. Users click on the Roo Code's setting button
2. Users create new profile and select Gemini as an API provider
3. Users see several custom parameters and configurations such as topP, topK, maxOutputTokens, enable URL context, enable Grounding Research and context limit.
   But I cannot see new parameters or configurations :(

Case 2: Strategically setting context limit / max context window

1. Users click on the Roo Code's setting button
2. Users create new profile and select Gemini as an API provider
3. Users change context limit / mac context window to aligned with their specific token usage
4. When users do multiple requests and Roo Code intelligently condense the context to be below the set context limit / max context window. Optional: If it goes over the set context limit, it will want useres
   But I am losing a lot of money :(

Enhancement of Gemini Integration and UX in Roo Code

Case 1: Embedding models

1. Users select Experimental section
2. Users check the Enable Codebase indexing
3. Users select Gemini as their Embedding Provider
4. Users select Gemini's embedding models
   But I cannot see Gemini as an Embedding Provider :(

Case 2: Set Thinking Budget

1. Users click on the Roo Code's setting button
2. Users create new profile and select Gemini as an API provider
3. Users selects newest Gemini 2.5 Pro Review
4. Instead of seeing Enable Reasoning, users see Set Thinking Budget
   But it is still confusing? Why do I have to enable reasoning if the new Gemini already has reasoning?

Case 3: Local Accurate Token Count

1. Users make an API request
2. Instead of sending another API request to get token count, it will be computed locally on user's machines
3. Users are happier because the waiting time is reduced :)
   But I am waiting forever for the request...

Estimated effort and complexity

Finer-grained control of Gemini models

Size: Medium (1-2 week)
Reasoning: Adding parameters and tools to genai functions - should not be difficult, UI/UX should not be difficult since it will be in the configuration profile, setting context limit (currently max context window) is already in progress
Main Challenges: optimal UI/UX designs
Dependencies: No new dependencies

Enhancement of Gemini Integration and UX in Roo Code

Size: Medium (1-2 weeks)
Reasoning: Minor UI/UX, token-counting should not be difficult
Main Challenges: complexity of integrating Gemini embedding model in Codebase Indexing
Dependencies: maybe more optimal token-counting libraries, could have some dependencies with this: #4375

Technical considerations (optional but helpful)

No response

Trade-offs and risks (optional)

No response

Additional context (optional)

No response

Proposal checklist

• I've searched existing Issues and Discussions for duplicates
• This is a specific, actionable proposal with clear problem and solution
• I've included concrete acceptance criteria
• I understand this needs approval before implementation begins

Interested in implementing this?

• Yes, I'd like to help implement this feature

[daniel-lxs]

Hey @HahaBill, the overall proposal looks good. It might be a good idea to split it into separate proposals with a narrower scope. That way, the PRs derived from them will be easier to test and integrate.

I'll move this proposal forward so that a dev team member can review it.

Let me know what you think!

[HahaBill]

Hey @HahaBill, the overall proposal looks good. It might be a good idea to split it into separate proposals with a narrower scope. That way, the PRs derived from them will be easier to test and integrate.

I'll move this proposal forward so that a dev team member can review it.

Let me know what you think!

Hi, thank you for reviewing this!! I also agree that this should be split in to separate issues/proposal. I made it this way so it looks comprehensive.

Anyway, I will split this into 2-3 issues and reference them to this proposal and I will let you guys know :)

[HahaBill]

@daniel-lxs Hi again! I partitioned this into 3 issues:

• Finer-grained control of Gemini models #4519
• Enhancement of Gemini Integration in Roo Code #4524
• Improving User Experience with Gemini in Roo Code #4526

[daniel-lxs]

I will mark this as not planned to keep the proposals organized