Fix: Vertex SDK system prompt caching compatibility #7711
Conversation
The Vertex SDK does not support array-form system prompts with cache_control, unlike the regular Anthropic SDK. This fix ensures the system parameter remains a plain string, applying cache_control only to message content blocks. - Modified system prompt handling to maintain string format - Updated tests to match corrected behavior - Maintains caching functionality for message blocks only Fixes compatibility issue introduced in commit a3d8c0e
dosubot
bot
added
size:S
daniel-lxs
moved this from PR [Needs Prelim Review]
to PR [Draft / In Progress]
in Roo Code Roadmap
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
There was a problem hiding this comment.
Thank you for this important fix! The changes correctly address the Vertex SDK compatibility issue by keeping system prompts as plain strings. The solution is clean, well-documented, and includes comprehensive test coverage.
The fix properly:
✅ Maintains system prompts as plain strings for Vertex SDK compatibility
✅ Updates the caching strategy documentation
✅ Includes thorough test coverage
✅ Preserves caching functionality for message content
Great work on identifying and fixing this compatibility issue!
| * 1. Maximum of 4 blocks can have cache_control | ||
| * 2. Only text blocks can be cached (images and other content types cannot) | ||
| * 3. Cache control can only be applied to user messages, not assistant messages | ||
| * 4. System prompt must remain as a plain string - Vertex SDK does not support array format |
There was a problem hiding this comment.
Excellent documentation update! The comment clearly explains the SDK limitation. Would it be worth adding a TODO about potentially re-enabling system prompt caching if future Vertex SDK versions support it? This could help future maintainers understand the limitation is SDK-specific rather than an API constraint.
| system: supportsPromptCache | ||
| ? [{ text: systemPrompt, type: "text" as const, cache_control: { type: "ephemeral" } }] | ||
| : systemPrompt, | ||
| // System must remain as plain string for Vertex SDK compatibility |
There was a problem hiding this comment.
Good change here. The comment clearly indicates this is for SDK compatibility rather than a design choice.
| cache_control: { type: "ephemeral" }, | ||
| }, | ||
| ], | ||
| system: "You are a helpful assistant", // System remains as plain string |
There was a problem hiding this comment.
Could we make the comment more explicit about why the system remains as a plain string? Something like: // System remains as plain string for Vertex SDK compatibility
| cache_control: { type: "ephemeral" }, | ||
| }, | ||
| ], | ||
| system: "You are a helpful assistant", // System remains as plain string |
There was a problem hiding this comment.
Same suggestion here - could we be more explicit about the SDK compatibility requirement in the comment?
github-project-automation
bot
moved this from PR [Draft / In Progress]
to Done
in Roo Code Roadmap
Summary
Fixes compatibility issue with Vertex SDK where system prompts were incorrectly converted to array format when caching is enabled.
Problem
The Vertex SDK does not support array-form system prompts with
cache_control, unlike the regular Anthropic SDK. When caching was enabled, the system prompt was being converted from a plain string to an array with cache metadata, causing compatibility issues.Solution
cache_controlonly to message content blocksChanges
src/api/providers/anthropic-vertex.tsto maintain system prompt as stringsrc/api/providers/__tests__/anthropic-vertex.spec.tsto match corrected behaviorRoot Cause
Issue introduced in commit a3d8c0e (April 28, 2025) as part of vertex/gemini prompt caching feature.
Fixes compatibility with
@anthropic-ai/vertex-sdkv0.7.0 and newer versions.Important
Fixes Vertex SDK compatibility by keeping system prompts as plain strings and applying
cache_controlonly to message content blocks.createMessage()andcompletePrompt()inanthropic-vertex.tsfor Vertex SDK compatibility.cache_controlapplied only to message content blocks, not system prompts.anthropic-vertex.spec.tsto reflect system prompt as string and verify cache control application.@anthropic-ai/vertex-sdkv0.7.0 and newer.This description was created by
for d98bd8f. You can customize this summary. It will automatically update as commits are pushed.