Skip to content

Commit bf35dcd

Browse files
fix: remove explicit cache_control for Google models in OpenRouter (#4487) (#4488)
* fix: remove explicit cache_control for Google models in OpenRouter (#4487) - Remove all Google models from OPEN_ROUTER_PROMPT_CACHING_MODELS set - This resolves 3+ minute lag when using google/gemini-2.5-pro-preview - OpenRouter still provides automatic implicit ephemeral caching for these models - Updated tests to handle intentional exclusion of Google models from explicit caching Fixes #4487 * refactor: simplify OpenRouter caching test logic - Replace hardcoded exclusion list with simple Google model filter - Keep original validation logic but make it more maintainable - Still ensures all our caching models are supported by OpenRouter - Still verifies we exclude all Google models from explicit caching * cleanup: remove unused excludedModels variable - Variable was defined but never used - Keeps the test logic clean and focused * refactor: only exclude google/gemini-2.5-pro-preview from caching - More surgical approach - only exclude the specific problematic model - Keep other Google models in caching (they work fine) - Add comment explaining the exclusion with issue reference - Update test to only exclude the specific model This targets just the model causing 3+ minute lag while preserving caching benefits for other Google models that work properly.
1 parent 736c4b1 commit bf35dcd

File tree

2 files changed

+16
-7
lines changed

2 files changed

+16
-7
lines changed

packages/types/src/providers/openrouter.ts

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,6 @@ export const OPEN_ROUTER_PROMPT_CACHING_MODELS = new Set([
3939
"anthropic/claude-3.7-sonnet:thinking",
4040
"anthropic/claude-sonnet-4",
4141
"anthropic/claude-opus-4",
42-
"google/gemini-2.5-pro-preview",
4342
"google/gemini-2.5-flash-preview",
4443
"google/gemini-2.5-flash-preview:thinking",
4544
"google/gemini-2.5-flash-preview-05-20",

src/api/providers/fetchers/__tests__/openrouter.spec.ts

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,12 +23,22 @@ describe("OpenRouter API", () => {
2323

2424
const models = await getOpenRouterModels()
2525

26-
expect(
27-
Object.entries(models)
28-
.filter(([_, model]) => model.supportsPromptCache)
29-
.map(([id, _]) => id)
30-
.sort(),
31-
).toEqual(Array.from(OPEN_ROUTER_PROMPT_CACHING_MODELS).sort())
26+
const openRouterSupportedCaching = Object.entries(models)
27+
.filter(([_, model]) => model.supportsPromptCache)
28+
.map(([id, _]) => id)
29+
30+
const ourCachingModels = Array.from(OPEN_ROUTER_PROMPT_CACHING_MODELS)
31+
32+
// Verify all our caching models are actually supported by OpenRouter
33+
for (const modelId of ourCachingModels) {
34+
expect(openRouterSupportedCaching).toContain(modelId)
35+
}
36+
37+
// Verify we have all supported models except intentionally excluded ones
38+
const excludedModels = new Set(["google/gemini-2.5-pro-preview"]) // Excluded due to lag issue (#4487)
39+
const expectedCachingModels = openRouterSupportedCaching.filter((id) => !excludedModels.has(id)).sort()
40+
41+
expect(ourCachingModels.sort()).toEqual(expectedCachingModels)
3242

3343
expect(
3444
Object.entries(models)

0 commit comments

Comments
 (0)