feat: vertex/gemini prompt caching

[ashktn]

Moving to @google/genai and reusing the gemini provider when using gemini on vertex ai

Context

Use the gemini provider when using a gemini model is selected on vertex ai.
@google/genai now support both Google Gemini and Vertex AI APIs
Prompt caching is already implemented in the gemini provider

Implementation

Used the gemini provider when a gemini model is selected on vertex ai.

Screenshots

Screen.Recording.2025-04-28.at.00.02.30.mov

How to Test

Select a gemini model that supports prompt caching on vertex ai. enable prompt caching

Get in Touch

ashktn

---

Important

Integrates Gemini prompt caching on Vertex AI by using GeminiHandler, updating models, and modifying tests and configurations.

• Behavior:
  • Integrates GeminiHandler for prompt caching with Gemini models on Vertex AI in vertex.ts and gemini.ts.
  • Updates vertexModels in api.ts to support prompt caching for Gemini models.
  • Removes @google-cloud/vertexai dependency from package.json.
• Testing:
  • Updates vertex.test.ts to mock GeminiHandler and test prompt caching behavior.
  • Removes vertex-gemini-format.test.ts and vertex-gemini-format.ts as they are no longer needed.
• Configuration:
  • Adds isVertex option to ProviderSettings in roo-code.d.ts, types.ts, and schemas/index.ts to distinguish between Gemini and Vertex AI usage.

^{This description was created by for f867837e867308b3e74f0cd15bf6701537d34b83. You can customize this summary. It will automatically update as commits are pushed.}

[changeset-bot]

🦋 Changeset detected

Latest commit: 77e4df6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
roo-cline	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[ellipsis-dev]

Consider wrapping JSON.parse(this.options.vertexJsonCredentials) in a try/catch block to handle potential JSON parsing errors. This improves resiliency if invalid credentials are provided.

^{This comment was generated because it violated a code review rule: mrule_OR1S8PRRHcvbdFib.}

[adamhill]

Thanks for tackling this difficult problem!

Even though you don't think it is perfect. We don't have to eat the elephant all in on bite. The whole troop can work on improving it now. 🦘❤️

Adding your comment from the channel Thread for completeness:

It seems like the right direction to use the @google/genai lib over @google-cloud/vertexai since google is encouraging using genai. The new lib says it works for Gemini 2.0, but i had no issues using Gemini 1.5

From a code structure perspective, it just didn't feel clean importing one provider from the other provider. As a future refactor task, we could move the common gemini code to a new file and then import that file in both providers

[mrubens]

@cte are you able to review this one? Thanks!

[cte]

I moved some stuff around; LMK if you any questions about my changes.

[ashktn]

I moved some stuff around; LMK if you any questions about my changes.

looks great!

[ashktn]

minor: seems like gemini-2.5-flash-preview-04-17 supports caching as per this doc- could be different between gemini vs vertex
https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview#supported_models