feat(claude): add remove-tools-cache-control config by whtiehack · Pull Request #1788 · router-for-me/CLIProxyAPI

whtiehack · 2026-03-02T05:30:16Z

Summary

add remove-tools-cache-control to claude-api-key entries
propagate the setting via synthesized auth attributes as remove_tools_cache_control=true
in Claude execute/stream flows, when enabled, strip tools[].cache_control
when stripping leaves no cache breakpoints, inject cache control for system/messages only (no tools injection)
update config.example.yaml
add tests for synthesizer attribute propagation and executor helper behavior

Testing

go test ./internal/runtime/executor ./internal/watcher/synthesizer
go build -o test-output ./cmd/server
go test ./... currently fails due to an unrelated pre-existing test in sdk/cliproxy/auth:
- TestManager_MaxRetryCredentials_LimitsCrossCredentialRetries

- add remove-tools-cache-control to Claude key config and example - propagate config through synthesized auth attributes - when enabled, strip tools.cache_control and avoid re-injecting tool cache control - add tests for synthesizer attribute propagation and executor helpers

gemini-code-assist · 2026-03-02T05:30:37Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the Claude API integration by introducing a new configuration option, remove-tools-cache-control. This feature addresses compatibility issues with upstream providers, such as AWS Bedrock proxies, that do not support tools.cache_control within tool definitions. By allowing the proxy to conditionally strip this specific cache control from requests, it ensures smoother operation and proper caching behavior while maintaining flexibility across different Claude API backends.

Highlights

New Configuration Option: Introduced a new remove-tools-cache-control boolean configuration option within the Claude API key settings to manage tools.cache_control behavior.
Attribute Propagation: Implemented logic to pass the remove-tools-cache-control setting as an attribute during the authentication synthesis process.
Conditional Cache Control Stripping: Modified Claude execution and streaming flows to conditionally strip tools.cache_control from request bodies when the new configuration is enabled.
Intelligent Cache Control Injection: Ensured that if tools.cache_control is stripped and no other cache breakpoints remain, cache_control is automatically injected for system and messages only, maintaining essential caching.
New Helper Functions and Tests: Added new helper functions (removeToolsCacheControl, stripToolsCacheControl) and comprehensive unit tests to validate the cache control removal and attribute handling logic.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

config.example.yaml
- Added a commented example for the remove-tools-cache-control configuration option under Claude key settings.
internal/config/config.go
- Defined the RemoveToolsCacheControl boolean field within the ClaudeKey struct to enable or disable stripping tools.cache_control.
internal/runtime/executor/claude_executor.go
- Modified the Execute and ExecuteStream methods to check for the removeToolsCacheControl flag from authentication attributes.
- Implemented conditional logic to call stripToolsCacheControl if the flag is enabled, and subsequently injectSystemCacheControl and injectMessagesCacheControl if no other cache controls are present.
- Added the removeToolsCacheControl helper function to determine if the remove_tools_cache_control attribute is enabled for a given authentication context.
- Added the stripToolsCacheControl helper function to remove cache_control fields from tool definitions within the request payload.
internal/runtime/executor/claude_executor_test.go
- Added TestRemoveToolsCacheControl to verify the correct parsing of the remove_tools_cache_control attribute from authentication contexts.
- Added TestStripToolsCacheControl to confirm that cache_control is successfully removed from tools while leaving system message cache controls untouched.
internal/watcher/synthesizer/config.go
- Updated the synthesizeClaudeKeys function to set the remove_tools_cache_control attribute to "true" in the coreauth.Auth object if RemoveToolsCacheControl is enabled in the ClaudeKey configuration.
internal/watcher/synthesizer/config_test.go
- Added an assertion to TestConfigSynthesizer_ClaudeKeys to ensure the remove_tools_cache_control attribute is correctly synthesized and present in the generated authentication object.

Activity

Unit tests for internal/runtime/executor and internal/watcher/synthesizer passed, confirming the functionality and configuration synthesis.
Noted pre-existing, unrelated test failures in sdk/cliproxy/auth.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a remove-tools-cache-control configuration for Claude providers. The changes are well-implemented across the configuration, synthesizer, and executor. I've identified a couple of areas for improvement regarding code duplication and error handling in the executor, which would enhance the maintainability and robustness of the new logic.

gemini-code-assist · 2026-03-02T05:32:19Z

internal/runtime/executor/claude_executor.go

+	if removeToolsCacheControl(auth) {
+		body = stripToolsCacheControl(body)
+		if countCacheControls(body) == 0 {
+			body = injectSystemCacheControl(body)
+			body = injectMessagesCacheControl(body)
+		}
+	} else if countCacheControls(body) == 0 {
 		body = ensureCacheControl(body)
 	}


This block of logic for handling cache_control is duplicated in ExecuteStream at lines 310-318. To improve maintainability and avoid potential inconsistencies, consider extracting this logic into a separate helper function. This function could then be called from both Execute and ExecuteStream.

internal/runtime/executor/claude_executor.go

luispater

Thanks for the PR. I’m requesting changes mainly around one correctness gap: the new remove-tools-cache-control behavior is applied in Execute/ExecuteStream, but not in CountTokens. A Claude provider configured for an upstream that rejects tools[].cache_control can still fail on /v1/messages/count_tokens because that path forwards tool cache_control unchanged. Could we apply the same stripping logic in CountTokens and add a regression test for a count_tokens request with remove_tools_cache_control=true? Once that is addressed, this looks good to me.

whtiehack · 2026-03-07T04:14:57Z

Done. Applied the same stripping logic in CountTokens and added a regression test (TestClaudeExecutor_CountTokens_RemoveToolsCacheControl). See 8c4ac39.

gemini-code-assist bot reviewed Mar 2, 2026

View reviewed changes

chore(claude): warn on tools cache_control strip errors

e8c41a7

luispater requested changes Mar 7, 2026

View reviewed changes

whtiehack requested a review from luispater March 7, 2026 04:06

fix(claude): apply remove-tools-cache-control in CountTokens

8c4ac39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(claude): add remove-tools-cache-control config#1788

feat(claude): add remove-tools-cache-control config#1788
whtiehack wants to merge 3 commits intorouter-for-me:mainfrom
whtiehack:remove_tools_cache

whtiehack commented Mar 2, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot commented Mar 2, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Mar 2, 2026

Uh oh!

Uh oh!

luispater left a comment

Uh oh!

whtiehack commented Mar 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

whtiehack commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Uh oh!

gemini-code-assist bot commented Mar 2, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

luispater left a comment

Choose a reason for hiding this comment

Uh oh!

whtiehack commented Mar 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

whtiehack commented Mar 2, 2026 •

edited

Loading