Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughAdded support for Google Cloud AI Platform (Vertex AI) and Google AI with Gemini models through a new Changes
Sequence DiagramsequenceDiagram
participant Client
participant Router as Request Router
participant AnthropicBuilder
participant GoogleBuilder
participant VertexAPI as Vertex API
participant GoogleAPI as Google AI API
Client->>Router: build_request(provider, model, prompt)
alt VertexAi + Anthropic (claude-*)
Router->>AnthropicBuilder: build_request()
AnthropicBuilder->>AnthropicBuilder: construct Anthropic request
AnthropicBuilder-->>Router: return Anthropic HTTP request
Router->>Router: rewrite URL to<br/>Vertex rawPredict/streamRawPredict
Router-->>Client: return Vertex-addressed request
else VertexAi + Gemini
Router->>GoogleBuilder: build_request()
GoogleBuilder->>GoogleBuilder: build Vertex URL<br/>convert prompt to GCP types<br/>serialize GenerateContentRequest
GoogleBuilder-->>Router: return HTTP request
Router-->>Client: return request (no auth)
else GoogleAi
Router->>GoogleBuilder: build_request()
GoogleBuilder->>GoogleBuilder: build Google AI URL<br/>convert prompt to GCP types<br/>serialize GenerateContentRequest
GoogleBuilder-->>Router: return HTTP request
Router-->>Client: return request (no auth)
end
Client->>VertexAPI: POST request
Client->>GoogleAPI: POST request
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
Merging this PR will not alter performance
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: e62a614d-9242-4350-b7d0-37b6cd05eb59
⛔ Files ignored due to path filters (1)
baml_language/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (5)
baml_language/Cargo.tomlbaml_language/crates/sys_llm/Cargo.tomlbaml_language/crates/sys_llm/src/build_request/google.rsbaml_language/crates/sys_llm/src/build_request/mod.rsbaml_language/crates/sys_llm/src/provider.rs
| Ok(format!( | ||
| "https://{domain}/v1/projects/{project_id}/locations/{location}/publishers/google/models/{model}:{rpc}" | ||
| )) |
There was a problem hiding this comment.
Incorrect publisher path for Anthropic models.
The URL template uses publishers/google for Anthropic/Claude models:
"https://{domain}/v1/projects/{project_id}/locations/{location}/publishers/google/models/{model}:{rpc}"Anthropic models on Vertex AI require publishers/anthropic in the path, not publishers/google. This will cause 404 errors when making requests to Claude models.
🐛 Proposed fix
Ok(format!(
- "https://{domain}/v1/projects/{project_id}/locations/{location}/publishers/google/models/{model}:{rpc}"
+ "https://{domain}/v1/projects/{project_id}/locations/{location}/publishers/anthropic/models/{model}:{rpc}"
))| LlmProvider::VertexAi if is_anthropic_on_vertex(client) => { | ||
| // Anthropic-on-Vertex needs the anthropic-version header. | ||
| // Vertex OAuth2 auth is not yet implemented here. | ||
| crate::auth_request::AnthropicAuth | ||
| .authorize(raw, client, &callbacks) | ||
| .await? | ||
| } | ||
| LlmProvider::VertexAi | LlmProvider::GoogleAi => { | ||
| // Auth not yet implemented -- return the raw request as-is. | ||
| raw | ||
| } |
There was a problem hiding this comment.
Missing authentication implementation for Vertex AI and Google AI.
The authorization section returns the raw request unchanged for non-Anthropic Vertex AI and Google AI providers. Per the context snippet from auth_request/mod.rs, only AnthropicAuth and OpenAiAuth implementations exist — there's no GoogleAuth or VertexAuth.
This means:
- Vertex AI requests will lack the required OAuth2 bearer token
- Google AI requests will lack the API key (typically passed as
?key=...query param)
Both providers will fail at runtime without proper authentication.
Would you like me to help implement the GoogleAuth handler that:
- For Google AI: appends the
api_keyas a query parameter (?key=...) - For Vertex AI: adds OAuth2 bearer token from credentials/ADC
| assert!(result.url.contains("claude-3-5-sonnet@20241022:rawPredict"),); | ||
| assert_eq!( | ||
| result.url, | ||
| "https://us-east5-aiplatform.googleapis.com/v1/projects/my-project/locations/us-east5/publishers/google/models/claude-3-5-sonnet@20241022:rawPredict" | ||
| ); |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Vertex AI Anthropic Claude rawPredict endpoint URL format
💡 Result:
The rawPredict endpoint URL format for Vertex AI Anthropic Claude models is: https://{LOCATION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{LOCATION}/publishers/anthropic/models/{MODEL}:rawPredict Where: - {LOCATION}: The region (e.g., 'us-central1', 'global' for global endpoint). - {PROJECT_ID}: Your Google Cloud project ID. - {MODEL}: The Claude model ID (e.g., 'claude-3-5-sonnet@20240620', 'claude-opus-4-6'). Use POST method with Authorization Bearer token and JSON body including "anthropic_version": "vertex-2023-10-16", messages, max_tokens, etc. For streaming, use :streamRawPredict instead.
Citations:
- 1: https://docs.anthropic.com/en/api/claude-on-vertex-ai
- 2: https://docs.cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.endpoints/rawPredict
- 3: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/claude/use-claude
- 4: https://minimusarchitecture.com/?_=%2Fvertex-ai%2Fgenerative-ai%2Fdocs%2Fpartner-models%2Fclaude%2Fuse-claude%23NpZ%2FcdvQaILjKBVh9YuxMiuGJWbHhSDB4mkVfbY%3D
- 5: https://platform.claude.com/docs/en/api/claude-on-vertex-ai
🏁 Script executed:
# Read the test file to verify the exact code at lines 1131-1135
sed -n '1100,1160p' baml_language/crates/sys_llm/src/build_request/mod.rsRepository: BoundaryML/baml
Length of output: 2226
🏁 Script executed:
# Also check the broader context of this test function to understand what it's testing
sed -n '1097,1155p' baml_language/crates/sys_llm/src/build_request/mod.rsRepository: BoundaryML/baml
Length of output: 2186
🏁 Script executed:
# Search for the implementation that constructs these URLs to understand the logic
rg "publishers" baml_language/crates/sys_llm/src/build_request/ -A 2 -B 2Repository: BoundaryML/baml
Length of output: 2996
🏁 Script executed:
# Check if there's conditional logic for Anthropic vs Google models
rg "anthropic" baml_language/crates/sys_llm/src/build_request/ -B 3 -A 3Repository: BoundaryML/baml
Length of output: 43215
🏁 Script executed:
# Look at the full google.rs implementation to see the build_url function context
cat -n baml_language/crates/sys_llm/src/build_request/google.rs | head -150Repository: BoundaryML/baml
Length of output: 6661
🏁 Script executed:
# Check the complete vertex_anthropic_url function
sed -n '89,123p' baml_language/crates/sys_llm/src/build_request/google.rsRepository: BoundaryML/baml
Length of output: 1329
Fix Vertex AI URL publisher for Anthropic models: use publishers/anthropic instead of publishers/google.
The Vertex AI URL for Anthropic models must use publishers/anthropic, not publishers/google. This issue exists in both the implementation and the test:
- Implementation (google.rs line 121): The
vertex_anthropic_url()function hardcodespublishers/googlein the URL template. - Test (lines 1131-1135): The assertion expects the incorrect URL with
publishers/google.
According to official Google Cloud and Anthropic documentation, the correct format is:
https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/publishers/anthropic/models/{model}:rawPredict
Using publishers/google will cause API requests to fail in production.
Binary size checks passed✅ 7 passed
Generated by |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
codeshaunted
force-pushed
the
avery/google
branch
from
bdeb626 to
dd853c1
Compare
1 participant
Summary by CodeRabbit