[Suggestion] Add ingress rate limiting and 403 circuit breaker for /v1/messages

[DarKWinGTM]

Summary
Ingress /v1/messages currently has no rate limiting or circuit breaker. Clients can send bursts directly to upstream; even when upstream returns 403 (e.g., CONSUMER_INVALID / SERVICE_DISABLED), requests continue rapidly, causing spam/flood and repeated error bursts.

Current behavior

• No per-IP/per-auth/per-model limits at Gin ingress; executors send straight via newProxyAwareHTTPClient.
• Central retry/backoff (sdk/cliproxy/auth/manager.go) defaults to retry=3, max interval=30s, which is still quick under client floods.
• No circuit breaker for persistent 403 (CONSUMER_INVALID, SERVICE_DISABLED); unusable keys/projects are retried immediately.

Expected behavior

• Rate-limit per IP + per auth/key + per model on /v1/messages.
• Persistent 403s are cooled down/blocked immediately, avoiding repeated upstream hits during the cooldown window.
• Retry/backoff for hard 4xx is lengthened or skipped to reduce spam.

Evidence / logs (examples)

• 403 PERMISSION_DENIED / CONSUMER_INVALID:
  • [2025-12-07 16:36:13] ... | 403 | ... | POST "/v1/messages?beta=true" body: "Permission denied on resource project calm-flow-0ffd9." metadata: consumer=projects/calm-flow-0ffd9, service=cloudaicompanion.googleapis.com
  • [2025-12-07 16:36:14] ... | 403 | ... | POST "/v1/messages?beta=true" body: "Permission denied on resource project calm-flow-80b89."
• 403 SERVICE_DISABLED (Gemini staging not enabled):
  • [2025-12-07 16:36:14] ... | 403 | ... | POST "/v1/messages?beta=true" body: "Gemini for Google Cloud API (Staging) has not been used in project firm-arcadia-b5j71 before or it is disabled..." metadata includes service=staging-cloudaicompanion.sandbox.googleapis.com, activationUrl=.../overview?project=firm-arcadia-b5j71
  • Similar burst for prime-turbine-hc2p6 in the same second.

Proposed changes

1. Gin middleware rate limiting

• Token bucket per IP + per auth/key + per model for /v1/messages (including ?beta=true).
• Return 429 before invoking executors.

2. Circuit breaker for persistent 403

• In auth manager: parse error body reason = CONSUMER_INVALID or SERVICE_DISABLED.
• Apply long cooldown (e.g., 5–10 minutes) per key/project/auth and reject locally during cooldown.

3. Adjust retry/backoff defaults

• Reduce/skip retries for these 403 classes.
• Keep config overrides for retry/backoff (backward compatible).

4. Observability (optional but useful)

• Metric/alert on error-rate per project/containerInfo; log reasons for block/cooldown.

Affected code (references)

• Executor direct send (no rate limit/CB): internal/runtime/executor/gemini_executor.go#L130-L147, internal/runtime/executor/gemini_executor.go#L215-L230
• Retry/backoff manager: sdk/cliproxy/auth/manager.go
• Retry config defaults: config.example.yaml

Acceptance criteria

• Rate limit middleware covers /v1/messages with buckets per IP + per auth/key + per model.
• 403 CONSUMER_INVALID / SERVICE_DISABLED are cooled down and not retried during cooldown.
• Retry/backoff for hard 4xx meaningfully reduces flood.
• Logs/metrics show block/cooldown reasons.

---

🤖 Analysis and report generated by Claude Code (gpt-5.1-codex-max-xhigh) via Deep Analysis Protocol.