feat: add Vertex AI support for Gemini models with user ADC#8
feat: add Vertex AI support for Gemini models with user ADC#8jwm4 wants to merge 3 commits intoredhat-et:mainfrom
Conversation
Add google-vertex provider for Gemini 2.5 Pro on Vertex AI, with a token-refresh sidecar that supports both service account keys and user Application Default Credentials (authorized_user type). Key changes: - Restructure setup.sh LLM prompts into a 3-choice menu - Add vertex-token-refresh sidecar that refreshes OAuth2 tokens and patches the gateway config with Bearer auth headers - Add google-vertex provider config with Authorization header for Vertex AI (works around OpenClaw sending x-goog-api-key which Vertex rejects) - Update docs to reflect Vertex AI setup and known anthropic-vertex limitation (requires image rebuild with vertex-sdk support) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughReworks setup and docs to prompt a single LLM source choice (Anthropic API key, Google Vertex AI, or local/in-cluster model), adds a Vertex AI token-refresh sidecar that writes Bearer tokens to a volume and patches the gateway, and expands provider/model config for Vertex (including Gemini). Changes
Sequence DiagramsequenceDiagram
participant User
participant SetupScript as setup.sh
participant Gateway
participant VertexSidecar as vertex-token-refresh
participant GoogleAPI as Google Vertex AI API
participant TokenVol as /vertex-token (emptyDir)
User->>SetupScript: Choose LLM source (Anthropic, Google Vertex, or local)
SetupScript->>SetupScript: Collect required inputs (project, region, SA path) for selected source
SetupScript->>Gateway: Deploy with env vars including VERTEX_AI_BASE_URL (if Vertex chosen)
rect rgba(100, 150, 200, 0.5)
Note over VertexSidecar,TokenVol: Runtime token refresh flow
VertexSidecar->>TokenVol: Create placeholder access-token.json
VertexSidecar->>GoogleAPI: Authenticate using SA or user creds
GoogleAPI-->>VertexSidecar: Return access token
VertexSidecar->>TokenVol: Write/update access-token.json
VertexSidecar->>Gateway: Patch gateway config with "Authorization: Bearer <token>"
Gateway->>TokenVol: Read access-token.json as needed
end
Gateway->>GoogleAPI: Call Vertex AI with Bearer token
GoogleAPI-->>Gateway: Model response
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
scripts/setup-agents.sh (1)
188-188:⚠️ Potential issue | 🟠 MajorMissing
${VERTEX_AI_BASE_URL}in ENVSUBST_VARS will break Vertex AI configuration.The
ENVSUBST_VARSlist here does not include${VERTEX_AI_BASE_URL}, butconfig-patch.yaml.envsubst(lines 69, 85) uses this variable for bothanthropic-vertexandgoogle-vertexprovider base URLs. Whensetup-agents.shprocesses templates, the literal${VERTEX_AI_BASE_URL}will remain unsubstituted, causing Vertex AI providers to have invalid URLs.
scripts/setup.sh(line 556) correctly includes this variable. This script should match.🐛 Proposed fix
Add the missing variable and compute
VERTEX_AI_BASE_URLbefore the envsubst loop:+# Vertex AI base URL: global endpoint has no region prefix +if [ "${GOOGLE_CLOUD_LOCATION:-}" = "global" ]; then + export VERTEX_AI_BASE_URL="https://aiplatform.googleapis.com" +else + export VERTEX_AI_BASE_URL="https://${GOOGLE_CLOUD_LOCATION:-us-central1}-aiplatform.googleapis.com" +fi + -ENVSUBST_VARS='${CLUSTER_DOMAIN} ${OPENCLAW_PREFIX} ${OPENCLAW_NAMESPACE} ${OPENCLAW_GATEWAY_TOKEN} ${OPENCLAW_OAUTH_CLIENT_SECRET} ${OPENCLAW_OAUTH_COOKIE_SECRET} ${ANTHROPIC_API_KEY} ${SHADOWMAN_CUSTOM_NAME} ${SHADOWMAN_DISPLAY_NAME} ${MODEL_ENDPOINT} ${DEFAULT_AGENT_MODEL} ${GOOGLE_CLOUD_PROJECT} ${GOOGLE_CLOUD_LOCATION} ${KEYCLOAK_URL} ${KEYCLOAK_REALM} ${KEYCLOAK_ADMIN_USERNAME} ${KEYCLOAK_ADMIN_PASSWORD} ${MLFLOW_TRACKING_URI} ${MLFLOW_EXPERIMENT_ID} ${MLFLOW_TLS_INSECURE}' +ENVSUBST_VARS='${CLUSTER_DOMAIN} ${OPENCLAW_PREFIX} ${OPENCLAW_NAMESPACE} ${OPENCLAW_GATEWAY_TOKEN} ${OPENCLAW_OAUTH_CLIENT_SECRET} ${OPENCLAW_OAUTH_COOKIE_SECRET} ${ANTHROPIC_API_KEY} ${SHADOWMAN_CUSTOM_NAME} ${SHADOWMAN_DISPLAY_NAME} ${MODEL_ENDPOINT} ${DEFAULT_AGENT_MODEL} ${GOOGLE_CLOUD_PROJECT} ${GOOGLE_CLOUD_LOCATION} ${VERTEX_AI_BASE_URL} ${KEYCLOAK_URL} ${KEYCLOAK_REALM} ${KEYCLOAK_ADMIN_USERNAME} ${KEYCLOAK_ADMIN_PASSWORD} ${MLFLOW_TRACKING_URI} ${MLFLOW_EXPERIMENT_ID} ${MLFLOW_TLS_INSECURE}'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@scripts/setup-agents.sh` at line 188, The ENVSUBST_VARS string in setup-agents.sh is missing ${VERTEX_AI_BASE_URL}, causing config-patch.yaml.envsubst to leave that placeholder unsubstituted; update the ENVSUBST_VARS constant to include ${VERTEX_AI_BASE_URL} and ensure VERTEX_AI_BASE_URL is computed/exported before the envsubst loop (same approach used in scripts/setup.sh) so templates (e.g., config-patch.yaml.envsubst) get the correct Vertex AI base URL at substitution time.agents/openclaw/overlays/openshift/config-patch.yaml.envsubst (1)
68-82:⚠️ Potential issue | 🔴 Critical
anthropic-vertexprovider uses incorrect authentication configuration for Vertex AI.Anthropic Claude on Vertex AI requires authentication via Google Cloud ADC (Application Default Credentials) and OAuth Bearer tokens, not an
apiKeyfield. The current configuration referencesGOOGLE_APPLICATION_CREDENTIALS(a file path to a service account JSON, not an API key value) which will fail authentication. Use the Bearer token approach shown in thegoogle-vertexprovider configuration instead.Additionally, the
anthropic-messagesadapter calls/v1/messages, while Vertex AI uses a different endpoint path, causing 404 errors.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@agents/openclaw/overlays/openshift/config-patch.yaml.envsubst` around lines 68 - 82, The antrop hic-vertex provider block uses an apiKey field and the wrong adapter path—replace the apiKey entry in the "anthropic-vertex" config with the same bearer/OAuth token style used by the existing google-vertex provider (i.e., remove "apiKey": {"source":"env","provider":"default","id":"GOOGLE_APPLICATION_CREDENTIALS"} and use a bearer token credential sourced from environment/OAuth ADC), and change the "api"/adapter to the Vertex-compatible messages endpoint (match the google-vertex adapter/path or the Vertex AI messages endpoint pattern instead of "anthropic-messages") so authentication uses ADC/OAuth bearer tokens and the request path matches Vertex AI.
🧹 Nitpick comments (1)
agents/openclaw/overlays/openshift/config-patch.yaml.envsubst (1)
39-44: Unused secrets provider:vertex-tokenis defined but not referenced.The
vertex-tokenfile provider is configured to read/vertex-token/access-token.json, but neitheranthropic-vertexnorgoogle-vertexactually references it. Thegoogle-vertexprovider uses a hardcoded placeholder inheaders.Authorizationwhich gets patched by the sidecar viased.This appears to be dead configuration. Either remove it, or if there's a plan to use it for a cleaner token injection approach, add a TODO comment.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@agents/openclaw/overlays/openshift/config-patch.yaml.envsubst` around lines 39 - 44, The "vertex-token" secrets provider under "secrets" -> "providers" is unused because neither "anthropic-vertex" nor "google-vertex" reference it (the latter uses a hardcoded placeholder patched via the sidecar in headers.Authorization); either delete the "vertex-token" provider entry to remove dead configuration, or leave it and add a clear TODO comment explaining it is intentionally reserved for a future cleaner token injection flow and referencing headers.Authorization, anthropic-vertex and google-vertex so reviewers know why it exists.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@agents/openclaw/base/openclaw-deployment.yaml.envsubst`:
- Around line 306-336: The sidecar loop patches CONFIG
("/home/node/.openclaw/openclaw.json") in-place with sed (in the while true loop
that refreshes TOKEN/TOKEN_FILE) but doesn't notify the gateway to reload its
config, so updated Bearer tokens may not be picked up; update the script after
the sed patch step to either (1) trigger the gateway to reload (e.g., send the
appropriate signal to the gateway process or invoke its reload endpoint) using
the gateway PID or management API if supported, or (2) if reload is unsupported,
add a clear comment near the CONFIG/TOKEN_FILE handling documenting this
limitation and the need to restart/reload the gateway for token changes to take
effect; reference the loop that sets TOKEN, the CONFIG variable, and the sed
patch operation when making the change.
---
Outside diff comments:
In `@agents/openclaw/overlays/openshift/config-patch.yaml.envsubst`:
- Around line 68-82: The antrop hic-vertex provider block uses an apiKey field
and the wrong adapter path—replace the apiKey entry in the "anthropic-vertex"
config with the same bearer/OAuth token style used by the existing google-vertex
provider (i.e., remove "apiKey":
{"source":"env","provider":"default","id":"GOOGLE_APPLICATION_CREDENTIALS"} and
use a bearer token credential sourced from environment/OAuth ADC), and change
the "api"/adapter to the Vertex-compatible messages endpoint (match the
google-vertex adapter/path or the Vertex AI messages endpoint pattern instead of
"anthropic-messages") so authentication uses ADC/OAuth bearer tokens and the
request path matches Vertex AI.
In `@scripts/setup-agents.sh`:
- Line 188: The ENVSUBST_VARS string in setup-agents.sh is missing
${VERTEX_AI_BASE_URL}, causing config-patch.yaml.envsubst to leave that
placeholder unsubstituted; update the ENVSUBST_VARS constant to include
${VERTEX_AI_BASE_URL} and ensure VERTEX_AI_BASE_URL is computed/exported before
the envsubst loop (same approach used in scripts/setup.sh) so templates (e.g.,
config-patch.yaml.envsubst) get the correct Vertex AI base URL at substitution
time.
---
Nitpick comments:
In `@agents/openclaw/overlays/openshift/config-patch.yaml.envsubst`:
- Around line 39-44: The "vertex-token" secrets provider under "secrets" ->
"providers" is unused because neither "anthropic-vertex" nor "google-vertex"
reference it (the latter uses a hardcoded placeholder patched via the sidecar in
headers.Authorization); either delete the "vertex-token" provider entry to
remove dead configuration, or leave it and add a clear TODO comment explaining
it is intentionally reserved for a future cleaner token injection flow and
referencing headers.Authorization, anthropic-vertex and google-vertex so
reviewers know why it exists.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ed6c3bf6-40a7-48b7-b1d0-e5007da9bf0f
📒 Files selected for processing (7)
README.mdagents/openclaw/README.mdagents/openclaw/base/openclaw-deployment.yaml.envsubstagents/openclaw/overlays/openshift/config-patch.yaml.envsubstdocs/TEAMMATE-QUICKSTART.mdscripts/setup-agents.shscripts/setup.sh
| while true; do | ||
| TOKEN="" | ||
| if [ "$CRED_TYPE" = "service_account" ]; then | ||
| # Service account key: activate then print token | ||
| if gcloud auth activate-service-account --key-file="$CRED" 2>/dev/null; then | ||
| TOKEN=$(gcloud auth application-default print-access-token 2>/dev/null) || true | ||
| fi | ||
| elif [ "$CRED_TYPE" = "authorized_user" ]; then | ||
| # User ADC: set GOOGLE_APPLICATION_CREDENTIALS and print token directly | ||
| TOKEN=$(gcloud auth application-default print-access-token 2>/dev/null) || true | ||
| else | ||
| echo "Unsupported credential type: ${CRED_TYPE:-empty}. Need service_account or authorized_user." | ||
| sleep 86400 | ||
| exit 0 | ||
| fi | ||
| if [ -n "$TOKEN" ] && [ -n "$GOOGLE_CLOUD_PROJECT" ]; then | ||
| echo "{\"token\": \"$TOKEN\", \"projectId\": \"$GOOGLE_CLOUD_PROJECT\"}" > "$TOKEN_FILE" | ||
| chmod 600 "$TOKEN_FILE" | ||
| # Patch the live gateway config with the Bearer token as a plain string header. | ||
| # OpenClaw's sanitizeModelHeaders drops non-string header values, so file secret | ||
| # refs in headers don't work. We write the resolved string directly instead. | ||
| CONFIG="/home/node/.openclaw/openclaw.json" | ||
| if [ -f "$CONFIG" ]; then | ||
| sed -i "s|\"Authorization\": \"Bearer [^\"]*\"|\"Authorization\": \"Bearer $TOKEN\"|" "$CONFIG" | ||
| echo "Config patched with Bearer token" | ||
| fi | ||
| echo "Token refreshed at $(date -u +%H:%M:%SZ)" | ||
| else | ||
| echo "Token refresh failed at $(date -u +%H:%M:%SZ)" | ||
| fi | ||
| sleep 3000 |
There was a problem hiding this comment.
Race condition: gateway may read stale config after sidecar patches it.
The sidecar patches openclaw.json in-place with sed -i, but the gateway may have already loaded the config and won't pick up the change until restart. The comment at line 324-326 acknowledges this is a workaround, but there's no mechanism to signal the gateway to reload config after a token refresh.
For the initial startup this works (gateway starts after sidecar writes placeholder), but subsequent token refreshes (every 50 minutes) won't propagate to the gateway.
Consider adding a comment documenting this limitation, or if the gateway supports config reload signals, trigger one after patching. Would you like me to help explore options?
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@agents/openclaw/base/openclaw-deployment.yaml.envsubst` around lines 306 -
336, The sidecar loop patches CONFIG ("/home/node/.openclaw/openclaw.json")
in-place with sed (in the while true loop that refreshes TOKEN/TOKEN_FILE) but
doesn't notify the gateway to reload its config, so updated Bearer tokens may
not be picked up; update the script after the sed patch step to either (1)
trigger the gateway to reload (e.g., send the appropriate signal to the gateway
process or invoke its reload endpoint) using the gateway PID or management API
if supported, or (2) if reload is unsupported, add a clear comment near the
CONFIG/TOKEN_FILE handling documenting this limitation and the need to
restart/reload the gateway for token changes to take effect; reference the loop
that sets TOKEN, the CONFIG variable, and the sed patch operation when making
the change.
…provider - Add VERTEX_AI_BASE_URL computation and include it in ENVSUBST_VARS in setup-agents.sh so Vertex baseUrl is substituted correctly when running setup-agents.sh independently - Remove unused vertex-token secrets provider from config template (no longer referenced after switching to sidecar config patching) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@scripts/setup-agents.sh`:
- Around line 176-181: The template gets passed an empty GOOGLE_CLOUD_LOCATION
which causes a double slash in the generated URL; normalize the effective
location before running envsubst by exporting a concrete variable (e.g., set
EFFECTIVE_GOOGLE_CLOUD_LOCATION or overwrite GOOGLE_CLOUD_LOCATION to
"us-central1" when unset) and use that normalized variable when building
VERTEX_AI_BASE_URL and when invoking envsubst for
agents/openclaw/overlays/openshift/config-patch.yaml.envsubst so the
"locations/${...}" segment is never empty.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 30c7cc07-bae6-47f3-8953-4b38abfe2f7c
📒 Files selected for processing (2)
agents/openclaw/overlays/openshift/config-patch.yaml.envsubstscripts/setup-agents.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- agents/openclaw/overlays/openshift/config-patch.yaml.envsubst
Prevents malformed Vertex URL (locations//publishers) when GOOGLE_CLOUD_LOCATION is unset in the environment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 participant
Summary
setup.shLLM prompts into a 3-choice menu (Anthropic API key, Vertex AI, local model) instead of prompting for all options sequentiallygoogle-vertexprovider config for Gemini 2.5 Pro on Vertex AIvertex-token-refreshsidecar container that refreshes OAuth2 access tokens every 50 minutes and patches the gateway config withAuthorization: Bearerheadersauthorized_usertype) — the sidecar auto-detects credential typeHow it works
Vertex AI rejects API keys (
x-goog-api-key) and requires OAuth2 Bearer tokens. OpenClaw'sgoogle-generative-aiAPI type always sends credentials as API keys. The workaround:Authorization: Bearer <token>as a plain string headerKnown limitation
Anthropic Claude via Vertex (
anthropic-vertex) requires a code change in OpenClaw itself (sallyom/openclaw@726a796) that is not yet in the deployed image. Docs note this limitation.Test plan
teardown.sh --delete-env+setup.shwith Vertex AI option and user ADCauthorized_usercredentials and refreshes tokengemini-2.5-proworks in Control UI🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation