Enhancement: Support Dynamic Runtime Management of Provider Credentials and OAuth Profiles

[MasuRii]

Is your enhancement related to a problem? Provide a clear and concise description.

The current credential management relies primarily on:

• Static environment variables (e.g., OPENAI_API_KEY_1, GEMINI_API_KEY_1) set at deployment time.
• The interactive credential tool (python -m rotator_library.credential_tool) and TUI for initial setup.
• Local file storage for OAuth tokens (oauth_creds/) or manual export to environment variables for stateless deployments.

This works well for self-hosted local/VPS setups but creates significant friction when deploying to cloud/hosting platforms (Render, Railway, Vercel, Fly.io, etc.):

• The instance owner must manually collect, configure, and update all provider API keys and OAuth profiles upfront.
• OAuth flows are interactive (browser-based with callback ports), which are challenging or impossible in serverless/stateless environments without open ports or persistent storage.
• There is no built-in way for end-users of a publicly hosted instance to provide their own credentials (BYO keys/OAuth) for providers they have access to.
• Adding new keys or OAuth profiles requires redeployment, environment variable updates, or manual intervention, limiting flexibility for shared/public proxies.

Proposed Enhancement

Introduce dynamic runtime credential and OAuth profile management to complement the existing static methods, with the following features:

1. Runtime Addition/Updating via API or Web UI:

   • Admin-protected endpoints (e.g., /admin/credentials/add, /admin/credentials/list) to add, update, or remove API keys and OAuth profiles without restart/redeploy.
   • Optional simple web dashboard (built on the existing TUI logic or a lightweight framework) for credential management.

2. User-Provided (BYO) Credentials:

   • Support for clients to pass their own provider API keys via request headers (e.g., X-Provider-Key: sk-... for OpenAI) or query params, with fallback to server-configured keys.
   • Per-request OAuth token injection (e.g., via headers) for providers requiring it.
   • Optional virtual key system (inspired by LiteLLM Proxy) where users can generate scoped keys tied to specific providers/models.

3. Improved OAuth Handling for Hosted Deployments:

   • Automatic token refresh using stored refresh tokens (without requiring interactive flows post-initial setup).
   • Support for user-initiated OAuth flows via redirect URLs compatible with hosted environments (e.g., using the proxy's own domain as callback).
   • Secure persistent storage option (e.g., optional SQLite/Redis backend for tokens, encrypted at rest).

4. Backward Compatibility & Security:

   • Retain full support for existing .env-based static configuration as the default.
   • Mix static (admin/global) and dynamic (user-provided) credentials with configurable precedence (e.g., prefer user-provided if present).
   • Built-in safeguards: rate limiting/quota tracking per credential, audit logging, and role-based access (admin vs. user).

Use Cases & Benefits

• Public/shared hosted instances can support a wide range of providers without the owner needing to obtain/pay for every key.
• Users can bring their own high-limit/paid accounts for specific providers (e.g., Gemini paid tier, Anthropic, etc.).
• Easier multi-user/team deployments: members add their own credentials dynamically.
• Better suitability for serverless/cloud platforms where static env vars are limiting.
• Enhanced resilience: more keys available through community/user contributions without manual admin work.

Additional Notes

• Leverage existing LiteLLM integration and rotator_library for key rotation/failover.
• Start with API-key providers, then extend to OAuth-heavy ones (Gemini, Antigravity, iFlow).
• Consider optional database integration for persistence (while keeping in-memory/ephemeral as default).
• Security is critical—happy to discuss authentication methods (e.g., master key, JWT).

This would make the proxy much more flexible for hosted and multi-user scenarios while preserving its excellent local/self-hosted experience. Thanks for the great project! Open to feedback or helping with implementation details.

[mirrobot-agent]

@MasuRii Thank you for submitting this issue. I am now beginning my analysis and will report back shortly.

[mirrobot-agent]

Initial Analysis Report

Summary: This is a well-structured enhancement request proposing a significant architectural extension to support dynamic runtime credential management, user-provided (BYO) credentials, and improved OAuth handling for cloud/hosted deployments.

Issue Validation: Confirmed — This is a valid, non-duplicate enhancement request. The feature gap described is real and supported by codebase analysis.

Reproducibility Assessment: N/A — This is a feature request, not a bug report.

---

Root Cause Analysis

The current credential management architecture is designed primarily for self-hosted/local deployments with the following characteristics:

1. Static Configuration via Environment Variables (credential_manager.py:23-28, main.py:346-352)

   • API keys are discovered from environment variables at startup using the pattern <PROVIDER>_API_KEY_<N>
   • OAuth credentials are stored in oauth_creds/ directory or loaded from env vars like ANTIGRAVITY_1_ACCESS_TOKEN

2. Interactive-Only OAuth Setup (credential_tool.py:1265-1332)

   • OAuth flows require browser-based interaction (setup_new_credential())
   • The credential tool is a Rich-based TUI requiring local terminal access
   • No HTTP/API-based credential management endpoints exist

3. Startup-Time Discovery Only (main.py:402-561, client.py:136-154)

   • The CredentialManager.discover_and_prepare() method runs once at startup
   • The RotatingClient is initialized with credentials at startup and maintains them in memory
   • No mechanism exists to inject or modify credentials post-startup

4. No Per-Request Credential Override (main.py:840-920)

   • The /v1/chat/completions endpoint passes requests directly to RotatingClient.acompletion()
   • No header inspection for user-provided keys (e.g., X-Provider-Key)
   • All requests use server-configured credentials via the rotation system

---

Gap Analysis vs. Proposed Features

Proposed Feature	Current State	Implementation Complexity
Runtime credential add/update API	Not implemented	Medium - requires state management refactor
BYO credentials via headers	Not implemented	Low-Medium - header parsing + fallback logic
Per-request OAuth injection	Not implemented	Medium - token validation per-request
Automatic OAuth token refresh	Implemented via BackgroundRefresher	Already exists
Persistent storage (SQLite/Redis)	Uses JSON files only	Medium - abstraction layer needed
Rate limiting per credential	Partially exists via CooldownManager	Low - extend existing system

---

Suggested Labels

• enhancement — This is a feature request for new functionality
• needs discussion — Architectural decisions needed (security model, storage backends, API design)

---

Proposed Next Steps

1. Break down into sub-issues: This enhancement encompasses multiple independent features. Consider creating separate issues for:

   • Admin API endpoints for credential management
   • BYO credentials via request headers
   • OAuth callback handling for hosted environments
   • Optional persistent storage backend

2. Security design document: Before implementation, define:

   • Authentication for admin endpoints (JWT, master key, etc.)
   • Credential isolation between users
   • Audit logging requirements
   • Rate limiting/quota tracking per BYO credential

3. MVP approach: Start with the lowest-friction feature:

   • Per-request API key passthrough via X-Provider-Key header
   • This requires minimal architectural changes and provides immediate value

4. Architecture consideration: The existing RotatingClient architecture assumes credentials are known at startup. Dynamic credentials may require:

   • A credential registry that supports runtime updates
   • Thread-safe credential addition/removal
   • Optional: separate credential pools for admin vs. user-provided keys

---

Missing Information

No additional information is required from the issue filer. The proposal is comprehensive and well-documented.

---

This analysis was generated by an AI assistant.

[Mirrowel]

So last time I tried to solve the problem the block was that it Is not possible to do interactive auth on client oc for the remote, without forwarding ports to exactly what the remote has. That is a lot of work, so instead just added .env export.

Other problem here is that hosts might lack file system access, so any kind of permanent storage is impossible.

The endpoint idea does solve it somewhat for runtime, but not fully .

Automatic token refresh is a thing already though, and works.

Though, that all seems solvable, and I do already have use case for multiple API keys and ability to use it more than just for yourself.

This is a big undertaking though. With holidays approaching might not get any work for weeks on this.

[MasuRii]

So last time I tried to solve the problem the block was that it Is not possible to do interactive auth on client oc for the remote, without forwarding ports to exactly what the remote has. That is a lot of work, so instead just added .env export.

Other problem here is that hosts might lack file system access, so any kind of permanent storage is impossible.

The endpoint idea does solve it somewhat for runtime, but not fully .

Automatic token refresh is a thing already though, and works.

Though, that all seems solvable, and I do already have use case for multiple API keys and ability to use it more than just for yourself.

This is a big undertaking though. With holidays approaching might not get any work for weeks on this.

That's true, I just suddenly has this idea when talking to one of my workmate to centralized all keys, basically everyone that you want to include in their team to contribute their keys without the hassle and everyone can benefit with the shared proxy.