Many services cannot be configured via env variables

[ligix]

What's happening?

Many llm providers cannot be configured via env variables even though all of them should have authentication and base url exposed to the config.

OpenAI is an example of a correctly configurable service via the two env variables OPENAI_API_KEY and OPENAI_API_HOST.
Gemini allows configuration of the api key only via GEMINI_API_KEY but there is no GEMINI_API_HOST.
Z.AI supports no configuration whatsoever.

Where does this happen?

Docker deployment

Impact on your workflow

Medium - Workaround exists

Environment (if applicable)

No response

Additional context

The offending code is in src/modules/llms/server.
This is correct:

      let alibabaOaiKey = access.oaiKey || env.ALIBABA_API_KEY || '';
      const alibabaOaiHost = llmsFixupHost(access.oaiHost || env.ALIBABA_API_HOST || DEFAULT_ALIBABA_HOST, apiPath);

This is only partially correct (does not support configuration of the host):

  const geminiHost = llmsFixupHost(access.geminiHost || DEFAULT_GEMINI_HOST, apiPath);
  let geminiKey = access.geminiKey || env.GEMINI_API_KEY || '';

it should instead be:

  const geminiHost = llmsFixupHost(access.geminiHost || env.GEMINI_API_HOST || DEFAULT_GEMINI_HOST, apiPath);
  let geminiKey = access.geminiKey || env.GEMINI_API_KEY || '';

This is a completely incorrect:

      let zaiKey = access.oaiKey || '';
      const zaiHost = llmsFixupHost(access.oaiHost || DEFAULT_ZAI_HOST, apiPath);

Make sure to find all occurrences of this, these aren't the only wrong ones!

[github-actions]

Thanks for the thorough report and for identifying specific code locations — very helpful.

Assessment

Confirmed. There's an inconsistency across vendors in src/modules/llms/server/ where many providers are missing env var fallbacks for API host and/or API key. Here's the full audit:

Currently complete (both key + host env vars):

Vendor	Key env var	Host env var
OpenAI	OPENAI_API_KEY	OPENAI_API_HOST
Anthropic	ANTHROPIC_API_KEY	ANTHROPIC_API_HOST
Alibaba	ALIBABA_API_KEY	ALIBABA_API_HOST
Azure	AZURE_OPENAI_API_KEY	AZURE_OPENAI_API_ENDPOINT
LocalAI	LOCALAI_API_KEY	LOCALAI_API_HOST
Bedrock	IAM creds / bearer	BEDROCK_REGION

Missing host env var (key is present):

Vendor	Has key	Missing host	File & line
Gemini	GEMINI_API_KEY	needs GEMINI_API_HOST	gemini/gemini.access.ts:40
Deepseek	DEEPSEEK_API_KEY	needs DEEPSEEK_API_HOST	openai/openai.access.ts:156
Groq	GROQ_API_KEY	needs GROQ_API_HOST	openai/openai.access.ts:174
Mistral	MISTRAL_API_KEY	needs MISTRAL_API_HOST	openai/openai.access.ts:216
Moonshot	MOONSHOT_API_KEY	needs MOONSHOT_API_HOST	openai/openai.access.ts:233
OpenRouter	OPENROUTER_API_KEY	needs OPENROUTER_API_HOST	openai/openai.access.ts:336
Perplexity	PERPLEXITY_API_KEY	needs PERPLEXITY_API_HOST	openai/openai.access.ts:356
TogetherAI	TOGETHERAI_API_KEY	needs TOGETHERAI_API_HOST	openai/openai.access.ts:378
OpenPipe	OPENPIPE_API_KEY	needs OPENPIPE_API_HOST	openai/openai.access.ts:331 (hardcoded)

Missing both key + host env vars:

Vendor	Missing	File & line
Z.ai	ZAI_API_KEY + ZAI_API_HOST	openai/openai.access.ts:414-415
LM Studio	LMSTUDIO_API_KEY + LMSTUDIO_API_HOST	openai/openai.access.ts:192-193

Missing host env var (key hardcoded to default):

Vendor	Has key	Missing host	File & line
xAI	XAI_API_KEY	needs XAI_API_HOST	openai/openai.access.ts:409 (hardcoded, ignores access.oaiHost)

Ollama (special case):

Has OLLAMA_API_HOST but no API key env var — reasonable since Ollama typically runs locally without auth, though an OLLAMA_API_KEY could be useful for secured remote instances.

Fix

The fix requires changes in two files:

1. src/server/env.server.ts — Add missing env var declarations (e.g. DEEPSEEK_API_HOST, GEMINI_API_HOST, GROQ_API_HOST, MISTRAL_API_HOST, MOONSHOT_API_HOST, OPENROUTER_API_HOST, PERPLEXITY_API_HOST, TOGETHERAI_API_HOST, XAI_API_HOST, ZAI_API_KEY, ZAI_API_HOST, LMSTUDIO_API_HOST, LMSTUDIO_API_KEY, OPENPIPE_API_HOST, OLLAMA_API_KEY)

2. Access files — Add env.VENDOR_API_HOST fallback in the host resolution and env.VENDOR_API_KEY where missing. For example:

   // gemini.access.ts:40 - add GEMINI_API_HOST
   const geminiHost = llmsFixupHost(access.geminiHost || env.GEMINI_API_HOST || DEFAULT_GEMINI_HOST, apiPath);
   
   // openai.access.ts:156 - add DEEPSEEK_API_HOST
   const deepseekHost = llmsFixupHost(access.oaiHost || env.DEEPSEEK_API_HOST || DEFAULT_DEEPSEEK_HOST, apiPath);
   
   // openai.access.ts:414 - add ZAI_API_KEY
   let zaiKey = access.oaiKey || env.ZAI_API_KEY || '';

   Same pattern for all other vendors listed above. The xAI case also needs to use llmsFixupHost(access.oaiHost || env.XAI_API_HOST || DEFAULT_XAI_HOST, apiPath) instead of hardcoding.

This is a straightforward, mechanical fix following the existing pattern established by OpenAI/Anthropic/Alibaba/LocalAI.

[enricoros]

Thanks for the input @ligix - the issue is that with 20+ supported providers the env vars are too numerous (and overloaded). I need your help to figure out an alternative, scalable, secure way to perform this configuration. What can we do?

[ligix]

the env vars are too numerous (and overloaded)

why would this be an issue? Big-AGI supports many providers and as such if you want to configure them it's expected there is going to be many knobs.
You should also consider that env vars are completely optional, if you don't want to configure XAI you can just not supply the env var and have a smaller .env file.

I haven't looked into other issues but if you want to solve stuff like #803 at the same time moving to a config file could be more "organized" and opens the door to more through configuration of bigagi itself too (such as setting defaults for other settings too).

If the decision was up to me I'd add the env vars since it's an easy and acceptable fix and if in the future more through configuration is needed I'd move to a full blown config file.

[enricoros]

@ligix Imagine a user has 5 OpenAI compatible services, how do we set all the variables in a uniform way? This is because not only one can have a service configured, but can have many (e.g. many openai, or many localai, etc) services configured. We can figure out a convention to put credentials in multiple variables, but it seems fragile compared to a configuration file.

[ligix]

but can have many (e.g. many openai, or many localai, etc) services configured

Ah, I did not know it was already supported since I use a single endpoint per service.
Assuming that then yeah, a config file is overdue.

[enricoros]

@ligix Any reference/ideas for which kind of config file?

[enricoros]

@claude Go ahead and explore and rank options. Search for industry standards. Score and then rank for risk reduction, security, generalization, extensibility, capability to declare multiple instances of the same vendor, future proofness, ease of configuration, migration, standardization, learning barrier, etc. explore many options

[claude]

Claude finished @enricoros's task in 1m 44s —— View job

---

Configuration Architecture Analysis

After researching the current codebase, industry standards, and patterns from comparable projects (LiteLLM, LibreChat, Dify, Open WebUI), here's a comprehensive evaluation of configuration approaches for Big-AGI's multi-vendor LLM setup.

---

Current State

Big-AGI uses flat environment variables (env.server.ts via t3-env/Zod) with inconsistent coverage across 19 vendors. The backend exposes boolean capability flags (hasLlmOpenAI, etc.) to the frontend. Users can also configure vendors via the UI (stored in Zustand/IndexedDB). The system already supports instanceLimit per vendor (e.g., OpenAI allows up to 10 instances), but env vars can only configure one instance per vendor.

---

Options Evaluated

#	Option	Pattern
1	Fix env vars (status quo+)	Add missing _HOST/_KEY vars for all vendors
2	Indexed env vars	OPENAI_1_API_KEY, OPENAI_2_API_KEY, etc.
3	JSON env var	BIGAGI_PROVIDERS='[{"vendor":"openai","key":"sk-..."}]'
4	YAML config file (LiteLLM/LibreChat pattern)	bigagi.yaml with env var interpolation
5	TypeScript config file	bigagi.config.ts (Next.js-native pattern)
6	JSON config file	bigagi.json with schema validation
7	Database-stored config (Dify pattern)	Admin UI writes to DB, no file needed
8	Hybrid: env vars + optional config file	Config file extends/overrides env vars

---

Scoring Criteria (1-5 scale, 5 = best)

Criterion	Weight	Description
Risk reduction	High	Fewer misconfigurations, validation, fail-fast
Security	High	Secrets handling, exposure surface, encryption support
Multi-instance	High	Multiple instances of same vendor (the core ask)
Ease of config	High	How easy for a Docker/self-host user to set up
Migration	Medium	Effort to migrate from current env var approach
Extensibility	Medium	Adding new vendors, new config fields
Standardization	Medium	Alignment with industry norms
Future-proofness	Medium	Accommodates growth (more vendors, features, tiers)
Learning barrier	Low	How much users need to learn

---

Detailed Evaluation

Option 1: Fix env vars (status quo+)

Add missing DEEPSEEK_API_HOST, GEMINI_API_HOST, ZAI_API_KEY, etc.

Criterion	Score	Notes
Risk reduction	3	Validated by Zod, but flat namespace is error-prone at scale
Security	4	Env vars are the standard secrets mechanism; Docker/K8s native
Multi-instance	1	Cannot configure multiple instances of the same vendor
Ease of config	5	Copy-paste into .env - everyone knows env vars
Migration	5	No migration needed - just add the missing vars
Extensibility	2	Each new vendor = 2-3 new vars; 19 vendors = 40+ vars already
Standardization	4	12-factor app standard
Future-proofness	1	Doesn't scale; can't express structured config
Learning barrier	5	Zero learning curve
Weighted total		~2.9

Option 2: Indexed env vars

OPENAI_1_API_KEY, OPENAI_1_API_HOST, OPENAI_2_API_KEY, etc.

Criterion	Score	Notes
Risk reduction	2	Naming convention is fragile; gaps in numbering, typos
Security	4	Same as env vars
Multi-instance	3	Works, but awkward - how many indices to scan?
Ease of config	3	Naming convention needs documentation
Migration	4	Existing vars work as _1_ variants
Extensibility	2	Explosive var count (vendors x instances x fields)
Standardization	1	Non-standard; no major project uses this pattern
Future-proofness	1	Doesn't scale at all
Learning barrier	3	Must learn the numbering convention
Weighted total		~2.4

Option 3: JSON env var

BIGAGI_PROVIDERS='[{"vendor":"openai","apiKey":"sk-...","apiHost":"..."}]'

Criterion	Score	Notes
Risk reduction	3	Can validate with Zod, but JSON-in-env is fragile (escaping)
Security	3	Secrets embedded in a single env var - visible in ps, logs
Multi-instance	4	Array naturally supports multiple instances
Ease of config	2	JSON in env vars is painful (escaping, no comments, no line breaks)
Migration	3	Breaking change - must migrate all existing env vars
Extensibility	3	Schema can evolve, but JSON editing is error-prone
Standardization	2	Some projects do this, but widely considered an anti-pattern
Future-proofness	2	Limited by env var length limits and ergonomics
Learning barrier	3	JSON is universal but editing in env context is painful
Weighted total		~2.7

Option 4: YAML config file (LiteLLM/LibreChat pattern) ⭐

bigagi.yaml with env var interpolation (${OPENAI_API_KEY})

# bigagi.yaml
version: 1
providers:
  - vendor: openai
    name: "OpenAI Production"
    apiKey: "${OPENAI_API_KEY}"
    apiHost: "https://api.openai.com"
  - vendor: openai
    name: "OpenAI via Helicone"
    apiKey: "${OPENAI_API_KEY_HELICONE}"
    apiHost: "https://oai.hconeai.com"
  - vendor: gemini
    apiKey: "${GEMINI_API_KEY}"
  - vendor: ollama
    apiHost: "http://localhost:11434"

Criterion	Score	Notes
Risk reduction	5	Schema validation at startup; fail-fast; structured
Security	5	${VAR} interpolation - secrets stay in env vars/.env. Config file itself is safe to commit. Compatible with SOPS for encrypted values
Multi-instance	5	Array entries - add as many instances as needed
Ease of config	4	YAML is human-friendly; comments supported; good editor support
Migration	3	Need to build loader + maintain backward compat with env vars
Extensibility	5	New fields, new vendors, new sections - YAML handles it naturally
Standardization	5	Industry standard: LiteLLM, LibreChat, Docker Compose, K8s all use YAML
Future-proofness	5	Can grow to include app settings, defaults, feature flags, access control
Learning barrier	3	YAML is widely known but has gotchas (indentation, types)
Weighted total		~4.5

Option 5: TypeScript config file

bigagi.config.ts (like next.config.ts, tailwind.config.ts)

// bigagi.config.ts
import { defineConfig } from 'big-agi/config';
export default defineConfig({
  providers: [
    { vendor: 'openai', apiKey: process.env.OPENAI_API_KEY },
    { vendor: 'gemini', apiKey: process.env.GEMINI_API_KEY },
  ],
});

Criterion	Score	Notes
Risk reduction	5	Full TypeScript type checking; IDE autocomplete
Security	4	process.env for secrets; file itself is code, harder to audit
Multi-instance	5	Arrays work naturally
Ease of config	3	Requires TypeScript knowledge; needs build step
Migration	2	Significant implementation effort; must integrate with Next.js build
Extensibility	5	Full programmatic power - conditionals, imports, functions
Standardization	3	Common in JS ecosystem (Next.js, Vite) but not in AI/LLM tools
Future-proofness	4	Very flexible but ties to TypeScript/Node.js
Learning barrier	2	Non-trivial for Docker users who just want to set API keys
Weighted total		~3.6

Option 6: JSON config file

bigagi.json with JSON Schema validation

Criterion	Score	Notes
Risk reduction	4	JSON Schema validation; structured
Security	3	No native env var interpolation - must build custom ${VAR} support
Multi-instance	5	Arrays work naturally
Ease of config	3	No comments allowed; strict syntax; less forgiving than YAML
Migration	3	Similar effort to YAML
Extensibility	4	JSON Schema is powerful but verbose
Standardization	3	Common format but less used for this purpose than YAML
Future-proofness	4	Universal format
Learning barrier	4	JSON is universally known
Weighted total		~3.5

Option 7: Database-stored config (Dify pattern)

Admin UI configures providers; stored in PostgreSQL/MongoDB

Criterion	Score	Notes
Risk reduction	4	UI validation; no syntax errors possible
Security	4	Encrypted at rest in DB; but requires DB infrastructure
Multi-instance	5	Unlimited instances via UI
Ease of config	4	Point-and-click; but requires running app first (chicken-and-egg)
Migration	1	Requires database dependency - Big-AGI is local-first/no-DB-required
Extensibility	4	Schema-driven UI generation
Standardization	3	Common in SaaS, unusual for self-hosted local-first apps
Future-proofness	4	Scales well for managed deployments
Learning barrier	5	Just use the UI
Weighted total		~3.3

Option 8: Hybrid - env vars + optional config file ⭐

Env vars work as today (backward compat). Optional bigagi.yaml extends/overrides.

# bigagi.yaml (optional - env vars still work without this file)
version: 1
providers:
  # Override/extend what env vars define
  - vendor: openai
    name: "Production"
    apiKey: "${OPENAI_API_KEY}"        # references existing env var
  - vendor: openai
    name: "Development"
    apiKey: "${OPENAI_DEV_API_KEY}"    # second instance
  - vendor: ollama
    apiHost: "${OLLAMA_API_HOST}"      # still works via env var

Criterion	Score	Notes
Risk reduction	4	Validated when present; env vars validated by Zod as before
Security	5	Env vars for secrets; config file for structure. Both patterns work
Multi-instance	5	Config file enables multi-instance; env vars for single-instance
Ease of config	5	Zero breaking change - env vars still work for simple setups. Config file only needed for advanced use
Migration	5	No migration required - existing .env files work unchanged
Extensibility	5	Config file can grow; env vars remain stable
Standardization	4	Follows "convention over configuration" principle
Future-proofness	5	Progressive disclosure: simple (env vars) -> advanced (config file)
Learning barrier	5	Users learn only what they need - env vars for basics, YAML for power users
Weighted total		~4.8

---

Final Ranking

Rank	Option	Score	Recommendation
1	Option 8: Hybrid (env vars + optional YAML)	4.8	Recommended - zero breaking changes, progressive complexity
2	Option 4: YAML config file	4.5	Strong if willing to deprecate env vars
3	Option 5: TypeScript config	3.6	Over-engineered for this use case
4	Option 6: JSON config file	3.5	YAML is strictly better for human-edited config
5	Option 7: Database config	3.3	Conflicts with local-first architecture
6	Option 1: Fix env vars	2.9	Quick fix but doesn't solve multi-instance
7	Option 3: JSON env var	2.7	Anti-pattern
8	Option 2: Indexed env vars	2.4	Fragile, non-standard

---

Recommendation: Option 8 (Hybrid)

Why this wins across all criteria:

1. Zero migration risk - existing Docker users with .env files are unaffected
2. Progressive disclosure - simple users never see YAML; power users opt in
3. Multi-instance support - the config file naturally handles instanceLimit > 1
4. Industry-proven - LiteLLM and LibreChat both use YAML with env var interpolation
5. Security-sound - secrets stay in env vars; config file is safe to version-control
6. Future-proof - can later add app defaults, feature flags, access control, model overrides

Suggested implementation phases:

1. Phase 0 (quick win): Fix the missing env vars (Option 1) as a stopgap - this is mechanical and addresses the original issue immediately
2. Phase 1: Add optional bigagi.yaml loader with Zod validation. When present, its providers array creates vendor service instances at startup. Env vars remain the fallback.
3. Phase 2: Extend the config file with app-level settings (defaults, feature flags) as needs arise

Key design decisions to make:

• File format: YAML vs TOML (YAML has wider ecosystem support; TOML avoids indentation issues)
• Env var interpolation syntax: ${VAR} (shell-like, LibreChat) vs os.environ/VAR (LiteLLM) - recommend ${VAR} as it's more universal
• Config file location: project root (Docker mount-friendly) or ~/.config/big-agi/ (user-level)
• Validation: fail-fast on startup (LibreChat) vs warn-and-continue

[ligix]

@ligix Any reference/ideas for which kind of config file?

well I'm not familiar with the internal structure of bigagi nor with what you want to show on the UI but json or yaml should be widespread enough:

providers:
 openai:
   type: openai
   oaiKey: SOME_OPENAI_API_KEY

 localai-proxy:
   type: localai
   oaiHost: http://localhost:1234/api/v1

 gemini:
   type: gemini
   geminiKey: SOME_GEMINI_API_KEY

 gemini-local:
   type: gemini
   geminiHost: http://localhost:1234/api/v1

{
 "providers": {
   "openai": {
     "type": "openai",
     "oaiKey": "SOME_OPENAI_API_KEY"
   },
   "localai-proxy": {
     "type": "localai",
     "oaiHost": "http://localhost:1234/api/v1"
   },
   "gemini": {
     "type": "gemini",
     "geminiKey": "SOME_GEMINI_API_KEY"
   },
   "gemini-local": {
     "type": "gemini",
     "geminiHost": "http://localhost:1234/api/v1"
   }
 }
}

Currently while can configure multiple openai providers and give them a custom name so you might as well just load the entire Access Schema from the config file:

// src/modules/llms/server/openai/openai.access.ts
export const openAIAccessSchema = z.object({
  dialect: z.enum([
    'alibaba', 'azure', 'deepseek', 'groq', 'lmstudio',
    'localai', 'mistral', 'moonshot', 'openai', 'openpipe',
    'openrouter', 'perplexity', 'togetherai', 'xai', 'zai',
  ]),
  clientSideFetch: z.boolean().optional(), // optional: backward compatibility from newer server version - can remove once all clients are updated
  oaiKey: z.string().trim(),
  oaiOrg: z.string().trim(), // [OpenPipe] we have a hack here, where we put the tags stringified JSON in here - cleanup in the future
  oaiHost: z.string().trim(),
  heliKey: z.string().trim(),

  // [OpenRouter only] Debug/routing service-level settings
  orRequireParameters: z.boolean().optional(), // Only route to providers supporting all request params
});

// src/modules/llms/server/gemini/gemini.access.ts
export const geminiAccessSchema = z.object({
  dialect: z.enum(['gemini']),
  clientSideFetch: z.boolean().optional(), // optional: backward compatibility from newer server version - can remove once all clients are updated
  geminiKey: z.string(),
  geminiHost: z.string(),
  minSafetyLevel: GeminiWire_Safety.HarmBlockThreshold_enum,
});

[enricoros]

@claude see and evaluate @ligix comment above. Also, consider a transformation of the protocol where the server is fully aware of the access objects created by the configuration, and will declare the vendors to the client, without the current "hasX, hasY, hasZ.." pattern which is suboptimal at this point. Make sure that the server and client configured services don't conflict in terms of namespaces (e.g. a server-side openai and a client-side openai shall both exist), they support advanced server set configuration (denylist or whitelist for models, both on client and then enforced by the server), come with rescan hashes, are stable (meaning that client sides have to use the same compound model IDs that must be aligned across clients of those servers), and there's a plan for server-set vs client added services on the UI side (pre population, disablement vs deletion, etc). Please consider these and more requirements (we got over time and in other tickets) and beging untangling the problem.

[claude]

Claude finished @enricoros's task in 3s —— View job

---

Server-Declared Vendor Configuration - Architecture Analysis

• Audit current vendor declaration pattern (hasX flags, access schemas, env vars)
• Evaluate @ligix's config file proposal against requirements
• Design server-declared vendor protocol (replacing hasX pattern)
• Design namespace coexistence (server-side + client-side same vendor)
• Design model filtering (denylist/allowlist, server-enforced)
• Design stable model IDs and rescan hashing
• Design UI plan for server-set vs client-added services
• Produce comprehensive architecture proposal

---

Current State Analysis

After auditing the codebase, here's how vendor configuration currently works:

Current Architecture (simplified flow)

Server (env vars) ─── backend.router.ts ──→ { hasLlmOpenAI: true, hasLlmGemini: true, ... }
                          │                          │
                          │                          ▼
                          │               store-backend-capabilities.ts (client Zustand)
                          │                          │
                          ▼                          ▼
              openai.access.ts              reconfigureBackendModels.ts
              (resolves env fallback)        (auto-creates services for hasX=true vendors)
                                                     │
                                                     ▼
                                            store-llms.ts
                                            (DModelsService[] + DLLM[])

Key files and their roles:

File	Role
src/server/env.server.ts	Declares env vars via t3-env/Zod (flat, ~40 vendor vars)
src/modules/backend/backend.router.ts:44-79	listCapabilities tRPC procedure - returns 18 hasLlmX booleans + hashLlmReconfig
src/modules/backend/store-backend-capabilities.ts	Client-side Zustand store for backend caps (18 booleans + hash)
src/modules/llms/vendors/IModelVendor.ts:24-25	hasServerConfigKey / hasServerConfigFn on vendor definition
src/common/logic/reconfigureBackendModels.ts	On boot: for each vendor where hasServerConfigKey is true, auto-creates a service + fetches models
src/modules/llms/server/openai/openai.access.ts	Resolves access (key/host) with access.oaiKey || env.VENDOR_KEY || '' fallback pattern
src/common/stores/llms/store-llms.ts	Client store for DModelsService[] (configured services) and DLLM[] (models)

Model ID construction (llm.client.ts:97):

id: `${service.id}-${_clientIdWithVariant(d.id, d.idVariant)}`
// e.g. "openai-gpt-4o", "openai-1-gpt-4o" (second OpenAI instance)

Service ID construction (store-llms.ts:323-331):

// First service: "openai", second: "openai-1", third: "openai-2"

Current Problems

1. hasLlmX is boolean-only - can't describe what was configured (key only? host only? both?), can't declare multiple instances, can't carry metadata
2. No model filtering - server can't restrict which models are visible or usable
3. Service IDs are local-only - openai, openai-1 are assigned by client creation order, so two clients of the same server get different IDs if they add services in different order
4. Model IDs are service-ID-dependent - openai-gpt-4o vs openai-1-gpt-4o for the same model on different service instances - breaks cross-client consistency
5. No server vs client service distinction - can't tell if a service was created by the server or the user
6. Missing env vars - 11 vendors missing _HOST env var, 2 missing _KEY (per the issue audit)

---

Evaluation of @ligix's Config File Proposal

@ligix proposed loading access schemas directly from a config file:

providers:
  openai:
    type: openai
    oaiKey: SOME_OPENAI_API_KEY
  gemini-local:
    type: gemini
    geminiHost: http://localhost:1234/api/v1

Assessment: The idea of mapping config entries to access schemas is sound, but the proposal needs several extensions for Big-AGI's requirements:

Aspect	Assessment
Maps to existing access schemas	Good - reuses existing validation
Named instances (e.g., gemini-local)	Good - provides stable IDs
Multi-instance same vendor	Good - natural with named entries
Missing: model filtering	Needs allowModels/denyModels
Missing: service metadata	Needs label, hidden, lock flags
Missing: env var interpolation	Raw secrets in config file is a security risk
Missing: server enforcement	Config file is static; server needs runtime enforcement

The core idea is adopted below with the necessary extensions.

---

Proposed Architecture: Server-Declared Services

Design Principles

1. Server is the authority for server-declared services - the client cannot modify their credentials
2. Client-added services coexist - users can add their own services alongside server-declared ones
3. Stable IDs - server services use deterministic IDs; model IDs are stable across clients
4. Progressive disclosure - env vars work for simple cases, config file for advanced
5. Server enforcement - model filtering is validated server-side, not just UI cosmetics

Phase 0: Fix Missing Env Vars (stopgap)

Add the missing env vars as described in the issue assessment. Mechanical fix, no architecture changes. This unblocks Docker users immediately.

Phase 1: Server-Declared Services Protocol

Replace hasLlmX booleans with a structured service declaration.

1.1 New Server Response Type

// New: replaces the 18 hasLlmX booleans
interface ServerDeclaredService {
  // Identity
  id: string;                    // stable server-assigned ID, e.g. "srv-openai", "srv-gemini-prod"
  vendorId: ModelVendorId;       // "openai", "googleai", "anthropic", etc.
  label: string;                 // Display name, e.g. "OpenAI (Production)"

  // Capabilities (what the server configured)
  hasKey: boolean;               // server has an API key configured
  hasHost: boolean;              // server has a custom host configured

  // Model filtering (server-enforced)
  modelFilter?: {
    mode: 'allowlist' | 'denylist';
    patterns: string[];          // glob patterns: "gpt-4o*", "claude-*-sonnet*"
  };

  // UI behavior
  locked?: boolean;              // user cannot modify credentials (default: true for server services)
  hidden?: boolean;              // don't show in vendor list (pre-configured but invisible)
}

interface BackendCapabilities {
  // NEW: structured service declarations (replaces hasLlmX booleans)
  serverServices: ServerDeclaredService[];

  // KEPT: non-LLM capabilities
  hasDB: boolean;
  hasBrowsing: boolean;
  hasGoogleCustomSearch: boolean;
  hasVoiceElevenLabs: boolean;

  // KEPT: reconfig hash (extended to include config file hash)
  hashLlmReconfig: string;

  // KEPT
  build?: { gitSha?: string; pkgVersion?: string; timestamp?: string; };
}

1.2 Backward Compatibility Strategy

The migration is not breaking because:

• hasLlmX booleans can be derived from serverServices on the client
• hasServerConfigKey on IModelVendor can check serverServices.some(s => s.vendorId === vendor.id)
• Env vars continue to work as before - the server just reads them into ServerDeclaredService objects

// Compatibility shim (in store-backend-capabilities.ts)
get hasLlmOpenAI(): boolean {
  return this.serverServices.some(s => s.vendorId === 'openai');
}

1.3 Service ID Namespacing

Problem: A server-declared "openai" and a user-added "openai" must coexist.

Solution: Prefix-based namespacing:

Server-declared:  "srv-{configId}"    e.g. "srv-openai", "srv-openai-helicone"
Client-added:     "{vendorId}"        e.g. "openai", "openai-1"  (unchanged)

• Server service IDs are deterministic (same config = same ID across all clients)
• Client service IDs are local (existing behavior, unchanged)
• Model IDs derive from service ID: srv-openai-gpt-4o vs openai-gpt-4o

Config-derived ID generation:

// From env vars: use vendor ID
id: `srv-${vendorId}`  // "srv-openai", "srv-googleai"

// From config file: use config entry key
id: `srv-${configEntryKey}`  // "srv-openai-prod", "srv-gemini-local"

1.4 Model ID Stability

Current problem: Model ID = ${serviceId}-${modelRefId} where serviceId depends on creation order.

Solution: For server-declared services, the service ID is deterministic (see above), so model IDs are automatically stable:

• srv-openai-gpt-4o is the same on every client connected to this server
• srv-openai-prod-gpt-4o (from config file) is also stable

For client-added services, IDs remain local (existing behavior - this is expected since they're per-client).

Phase 2: Optional Config File

2.1 Config File Format

File: bigagi.yaml (or bigagi.json) in project root (Docker-mount-friendly).

# bigagi.yaml
version: 1

services:

  # Simple: just like setting OPENAI_API_KEY env var, but with a stable name
  openai:
    vendor: openai
    apiKey: "${OPENAI_API_KEY}"  # env var interpolation - secrets stay in .env

  # Multi-instance: two OpenAI endpoints
  openai-helicone:
    vendor: openai
    apiKey: "${OPENAI_API_KEY}"
    apiHost: "https://oai.hconeai.com"
    label: "OpenAI via Helicone"

  # Gemini with custom host (the original issue)
  gemini:
    vendor: gemini
    apiKey: "${GEMINI_API_KEY}"
    apiHost: "${GEMINI_API_HOST}"

  # Local LLM
  ollama-local:
    vendor: ollama
    apiHost: "http://gpu-server:11434"
    label: "Local Ollama"

  # Service with model filtering
  openrouter-restricted:
    vendor: openrouter
    apiKey: "${OPENROUTER_API_KEY}"
    label: "OpenRouter (Approved Models)"
    modelFilter:
      mode: allowlist
      patterns:
        - "anthropic/claude-*"
        - "openai/gpt-4o*"
        - "google/gemini-2*"

  # Lock a service: users can see models but can't change credentials
  anthropic-shared:
    vendor: anthropic
    apiKey: "${ANTHROPIC_API_KEY}"
    locked: true
    label: "Anthropic (Shared)"

2.2 Config Loading Priority

1. Config file (bigagi.yaml) — highest priority, structured
2. Env vars — fallback, simple single-instance per vendor
3. Client-side — user adds their own services via UI

When a config file exists:

• Config file entries become ServerDeclaredService objects
• Env vars that correspond to a vendor already in the config file are ignored (config file wins)
• Env vars for vendors NOT in the config file still work (backward compat)

When no config file exists:

• Current env var behavior is preserved exactly

2.3 Config-to-Access Mapping

Each config entry maps to the existing access schemas. The server reads the config file at startup and creates access resolvers:

// Mapping config keys to access schema fields
const CONFIG_TO_ACCESS: Record<ModelVendorId, Record<string, string>> = {
  openai:    { apiKey: 'oaiKey',     apiHost: 'oaiHost',     apiOrg: 'oaiOrg',     heliconeKey: 'heliKey' },
  googleai:  { apiKey: 'geminiKey',  apiHost: 'geminiHost' },
  anthropic: { apiKey: 'apiKey',     apiHost: 'apiHost' },
  ollama:    { apiKey: 'apiKey',     apiHost: 'apiHost' },
  // ... uniform field names in config, mapped to vendor-specific schema fields
};

Phase 3: Server-Enforced Model Filtering

3.1 Filter Application Points

Model filtering must be enforced at two points:

1. Model listing (soft) - filtered models don't appear in the client's model list
2. Request execution (hard) - server rejects requests for filtered models

// In the model listing tRPC procedure
function applyModelFilter(
  models: ModelDescription[],
  filter: ServerDeclaredService['modelFilter'],
): ModelDescription[] {
  if (!filter) return models;

  const matchesPattern = (modelId: string) =>
    filter.patterns.some(pattern => minimatch(modelId, pattern));

  return filter.mode === 'allowlist'
    ? models.filter(m => matchesPattern(m.id))
    : models.filter(m => !matchesPattern(m.id));
}

// In AIX request handler - server-side enforcement
function validateModelAccess(
  serviceId: string,
  modelRefId: string,
  serverServices: ServerDeclaredService[],
): void {
  const service = serverServices.find(s => s.id === serviceId);
  if (!service?.modelFilter) return;

  const matchesPattern = (id: string) =>
    service.modelFilter!.patterns.some(p => minimatch(id, p));

  const allowed = service.modelFilter.mode === 'allowlist'
    ? matchesPattern(modelRefId)
    : !matchesPattern(modelRefId);

  if (!allowed)
    throw new TRPCError({ code: 'FORBIDDEN', message: `Model ${modelRefId} is not allowed on service ${serviceId}` });
}

3.2 Client-Side Filter Awareness

The client receives the filter as part of ServerDeclaredService. This allows:

• Hiding filtered models in the UI without a round-trip
• Showing a "restricted" indicator on server services with filters
• Greying out models that don't match the filter

Phase 4: UI Design for Server vs Client Services

4.1 Service List Behavior

┌─ Models Setup ────────────────────────────┐
│                                           │
│  SERVER-CONFIGURED                        │
│  ┌──────────────────────────────────────┐ │
│  │ 🔒 OpenAI (Production)    [Refresh] │ │ <- locked: can't edit credentials
│  │ 🔒 Anthropic (Shared)     [Refresh] │ │ <- locked: credentials hidden
│  │    Ollama (Local)          [Refresh] │ │ <- not locked: user can adjust host
│  └──────────────────────────────────────┘ │
│                                           │
│  YOUR SERVICES                            │
│  ┌──────────────────────────────────────┐ │
│  │    OpenAI (Personal)       [Delete]  │ │ <- user-added, fully editable
│  │    Groq                    [Delete]  │ │
│  └──────────────────────────────────────┘ │
│                                           │
│  [+ Add Service]                          │
│                                           │
└───────────────────────────────────────────┘

4.2 Server Service UI Rules

Property	Server Service (locked)	Server Service (unlocked)	Client Service
Credentials	Hidden, non-editable	Shown, non-editable	Editable
Label	Server-set, non-editable	Editable	Editable
Delete	Cannot delete	Cannot delete	Can delete
Models visibility	Can toggle hidden	Can toggle hidden	Can toggle hidden
Model filter	Shown as read-only badge	Shown as read-only badge	N/A
Refresh models	Yes	Yes	Yes

4.3 Pre-population

On first load (or when config changes):

1. Server services are auto-created with srv- prefix IDs
2. Models are auto-fetched for all server services
3. If the user already has a client-side service for the same vendor, both coexist
4. Server services appear first in the list, grouped under "Server-Configured"

4.4 Service Removal

• Server services cannot be deleted by the client - they reappear on refresh
• If the deployer removes a service from the config file, the server no longer declares it
• On next reconfigureBackendModels, the orphaned server service is cleaned up
• Client services can always be deleted (existing behavior)

Phase 5: Rescan Hash Evolution

The current hashLlmReconfig (in backend.router.ts:23-35) hashes env var values + AIX monotonic version.

Extended hash includes:

function generateLlmEnvConfigHash(env: Record<string, unknown>, configFile?: ParsedConfig): string {
  const hashInputs = [
    Release.Monotonics.Aix.toString(),
    Release.TenantSlug.toString(),
    // env vars (existing)
    ...envAPIKeys,
    // config file hash (new)
    configFile ? JSON.stringify(configFile) : '',
  ];
  return sdbmHash(hashInputs.join(';'));
}

This ensures the client re-fetches models when:

• Env vars change (existing)
• Config file changes (new)
• AIX protocol version changes (existing)

---

Implementation Roadmap

Phase 0: Fix Missing Env Vars (1-2 hours, no arch changes)

• Add missing _HOST and _KEY env vars to env.server.ts
• Add env fallbacks in access files
• Add missing hasLlmX entries (ZAI, LMStudio)
• Ships independently, unblocks Docker users

Phase 1: Server-Declared Services Protocol (medium, ~2-3 days)

1. Define ServerDeclaredService type
2. Replace hasLlmX booleans with serverServices[] in backend router
3. Add backward-compat shim in store-backend-capabilities.ts
4. Update reconfigureBackendModels.ts to use serverServices
5. Update IModelVendor.hasServerConfigKey consumers
6. Implement srv- prefix namespacing for server service IDs
7. Update vendor setup UIs for locked/server-managed state

Phase 2: Config File Support (medium, ~2-3 days)

1. Add YAML parser dependency (e.g., yaml)
2. Implement config file loader with env var interpolation
3. Implement config-to-ServerDeclaredService mapping
4. Add config file hash to reconfig hash
5. Document config file format and examples

Phase 3: Model Filtering (small-medium, ~1-2 days)

1. Add modelFilter to ServerDeclaredService
2. Implement filter in model listing response
3. Implement server-side enforcement in AIX request handler
4. Add filter UI indicators on client

Phase 4: UI Polish (small-medium, ~1-2 days)

1. Split service list into "Server" and "Your Services" sections
2. Implement locked service UI (hidden credentials, no delete)
3. Show model filter badges
4. Handle service lifecycle (auto-cleanup of removed server services)

---

Key Design Decisions Needed

1. Config file format: YAML vs TOML vs JSON? (Recommendation: YAML - industry standard for LLM tools, comments supported, good Docker/K8s ecosystem fit)

2. Config file location: Project root (./bigagi.yaml) vs environment-specified path (BIGAGI_CONFIG_FILE)? (Recommendation: project root with env override)

3. Env var interpolation syntax: ${VAR} (shell-like, used by Docker Compose, LibreChat) vs other? (Recommendation: ${VAR} - most universal)

4. Backward compat timeline: Keep hasLlmX forever as derived properties, or deprecate after N releases? (Recommendation: keep as derived - zero cost, prevents breakage)

5. Model ID migration: When a user upgrades and server services get srv- prefixed IDs, their existing model assignments (chat model, fast model) will have old IDs. Need migration logic in onRehydrateStorage. (Recommendation: auto-migrate openai-gpt-4o -> srv-openai-gpt-4o when a matching server service exists)

6. Glob library for model patterns: minimatch (widely used) vs picomatch (faster, zero-dep)? (Recommendation: picomatch - smaller bundle, used in server-only code)

---

[enricoros]

@claude let's do YAML, no migration (hijacking/security reasons), need the model id construction to be parsed unambiguously (see the - in model names, :: in variants, etc.. it's messy, need to be crystal clear on both ends), this can replace the old system for new deployments, shall have a per-service monotonic in addition to the global hash to force refresh. See all other app's formats and check if there are important traits or features we miss. Make sure in our file this is for LLM/AIX purpose, but we may also have Speex or other configs and free services in the future, for instance Browsing comes to mind and it's already real. Consider this from a high level security / authorization perspective too: enable the client to perform all the operations via UIDs and resource/function authorization without access to the underlying data. Feel free to sketch a reusable framework for it. Don't make this too overfit.

[claude]

Claude finished @enricoros's task in 8m 15s —— View job

---

Server Configuration Architecture - YAML Design

• Audit current model ID construction, service IDs, variant handling
• Research competitor config formats (LiteLLM, LibreChat, Dify, Open WebUI, Lobe Chat, Jan, Portkey, Cloudflare, vLLM)
• Design unambiguous model ID scheme
• Design YAML schema (multi-purpose: LLM, Speex, Browsing, future services)
• Design per-service monotonic + global hash refresh mechanism
• Design security/authorization framework (UID-based, no secret exposure to client)
• Final write-up

---

Current State Audit

Model ID Construction (problematic)

Current code at src/modules/llms/llm.client.ts:97:

id: `${service.id}-${_clientIdWithVariant(d.id, d.idVariant)}`

Where _clientIdWithVariant (llm.client.ts:21-25):

// idVariant starts with "::" -> no dash separator
// otherwise -> dash separator
function _clientIdWithVariant(id: string, idVariant?: string): string {
  return !idVariant ? id
    : idVariant.startsWith('::') ? `${id}${idVariant}`
      : `${id}-${idVariant}`;
}

Current ID examples:

• openai-gpt-4o (service=openai, model=gpt-4o)
• openai-1-gpt-4o-2024-11-20 (service=openai-1, model=gpt-4o-2024-11-20)
• Clone: openai-gpt-4o::my-variant (uses :: separator)

Ambiguity problem: Given openai-1-gpt-4o-2024-11-20, you cannot tell where the service ID ends and the model ref begins. Is it openai-1 + gpt-4o-2024-11-20? Or openai + 1-gpt-4o-2024-11-20? Both - in service IDs and - in model IDs make parsing ambiguous.

Service ID Construction (fragile)

Current code at store-llms.ts:323-331:

// "openai", "openai-1", "openai-2" - first available slot
function _locallyUniqueServiceId(vendorId, existingServices): DModelsServiceId {
  let serviceId = vendorId;
  let serviceIdx = 0;
  while (existingServices.find(s => s.id === serviceId)) {
    serviceIdx++;
    serviceId = `${vendorId}-${serviceIdx}`;
  }
  return serviceId;
}

Client-local, non-deterministic (order-dependent), uses - as separator.

---

Competitor Config Format Research

Comprehensive audit of 10 tools in the LLM gateway/proxy space:

LiteLLM (litellm_config.yaml)

model_list:
  - model_name: gpt-4o              # public alias
    litellm_params:
      model: azure/my-deployment     # vendor/model routing
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
router_settings:
  routing_strategy: least-busy       # 4 load balancing strategies
general_settings:
  master_key: sk-...

Key traits: os.environ/VAR interpolation, 4 routing strategies (simple-shuffle, least-busy, usage-based, latency-based), model aliasing (many-to-one), runtime model add via /model/new API, S3/GCS config loading, hierarchical RBAC (Org -> Team -> User -> Key), per-provider budget caps.

LibreChat (librechat.yaml)

version: 1.2.1
endpoints:
  custom:
    - name: "Mistral"
      apiKey: "${MISTRAL_API_KEY}"
      baseURL: "https://api.mistral.ai/v1"
      models:
        default: ["mistral-large-latest"]
        fetch: true
      dropParams: ["stop"]

Key traits: ${VAR} env interpolation, Zod-validated config with fail-fast (exit code 1 on validation failure), models.fetch: true for dynamic discovery, dropParams for vendor quirks, modelSpecs with enforce: true for strict model control and group-based UI organization. RBAC + resource-level permissions. UI-created configs intentionally block server-side env var access for security.

LobeChat (env vars only)

Notable: Powerful model list DSL - +model to add, -model to hide, -all to clear defaults, model->deployName=displayName for aliasing. Example: OPENAI_MODEL_LIST=-all,+gpt-4o,+o3. Multi-key rotation with random or turn (round-robin) modes. No config file, no multi-instance (feature requested).

Open WebUI (env vars + DB)

Notable: PersistentConfig dual-layer system - DB values override env vars by default; ENABLE_PERSISTENT_CONFIG=False forces env-var-only mode. Per-model Private/Public toggle with group/user ACLs. DEFAULT_MODEL_METADATA for baseline capabilities. Model "wrapping" - bind system prompts, knowledge, tools to any model.

Portkey AI Gateway (JSON configs)

Notable: Virtual key vault - store provider API keys in Portkey, reference by virtual key ID with rotation/revocation. targets array with weight for load balancing. fallback and loadbalance routing modes. Semantic caching. Guardrails (50+). Canary deployments.

Others (Dify, Jan, Ollama, vLLM, Cloudflare)

• Dify: DB-driven, POSITION_PROVIDER_PINS/EXCLUDES for UI ordering
• Jan: YAML per model (model.yml), desktop-first
• Ollama: Modelfile format, env vars only, community requesting YAML config (#2941)
• vLLM: YAML for CLI args, single model per server instance
• Cloudflare AI Gateway: Dashboard/API config, AES-encrypted Secrets Store, dynamic routing

Comparison Matrix

Feature	LiteLLM	LibreChat	Open WebUI	LobeChat	Portkey
Config format	YAML	YAML	Env+DB	Env only	JSON
Multi-instance same vendor	Yes (native)	Yes (custom)	Env vars	No	Yes (targets)
Model filtering	Access groups	modelSpecs+enforce	Admin toggle	DSL syntax	Via config
Env var interpolation	os.environ/	${ENV_VAR}	Direct	Direct	Virtual keys
Schema validation	Partial	Zod (fail-fast)	None	TypeScript	None
RBAC	Org/Team/User/Key	RBAC+perms	Per-model ACLs	Password	Key-level
Load balancing	4 strategies	None	None	Key rotation	Weighted
Runtime updates	/model/new API	CONFIG_PATH reload	PersistentConfig	None	Config ID

---

Proposed Architecture

Design Principles

1. YAML, no migration - new system for new deployments. Old env vars remain functional independently.
2. Unambiguous IDs - use a delimiter that cannot appear in vendor IDs or model ref IDs
3. Multi-purpose - LLMs, Speex, Browsing, future services share the same framework
4. UID-based authorization - client never sees secrets, operates via opaque service UIDs
5. Per-service monotonic - each service has its own version counter for targeted refresh
6. Not overfit - reusable framework, not LLM-specific

1. Unambiguous Model ID Scheme

Problem: - appears in both service IDs (openai-1) and model ref IDs (gpt-4o-2024-11-20), making compound IDs ambiguous.

Solution: Use @ as the service-model separator.

Format:  {serviceUid}@{modelRefId}
         {serviceUid}@{modelRefId}::{cloneVariant}

Examples:
  openai@gpt-4o
  my-prod-openai@gpt-4o-2024-11-20
  srv.gemini@gemini-2.0-flash
  srv.openai-prod@gpt-4o::{my-custom-variant}

Why @:

• Not used in any vendor model ID across all 19 vendors (model IDs use -, /, ., :)
• Visually reads as "model AT service" - intuitive
• Single character, easy to split: id.split('@') always gives exactly [serviceUid, modelRefWithVariant]
• :: remains the clone variant separator (already established), and is unambiguous since :: doesn't appear in model ref IDs

Service UID rules:

• Server-declared services: srv.{configKey} (e.g., srv.openai, srv.openai-prod, srv.gemini-local)
• Client-added services: {vendorId} or {vendorId}.{n} (e.g., openai, openai.2)
• The . as instance separator within service UIDs (replacing - for instance numbering) avoids collision with vendor IDs that contain -

Parsing is always unambiguous:

function parseModelId(dllmId: string): { serviceUid: string; modelRef: string; cloneVariant?: string } {
  const atIdx = dllmId.indexOf('@');
  if (atIdx === -1) throw new Error(`Invalid model ID: ${dllmId}`);
  const serviceUid = dllmId.slice(0, atIdx);
  const rest = dllmId.slice(atIdx + 1);
  const variantIdx = rest.indexOf('::');
  return variantIdx === -1
    ? { serviceUid, modelRef: rest }
    : { serviceUid, modelRef: rest.slice(0, variantIdx), cloneVariant: rest.slice(variantIdx + 2) };
}

2. YAML Config Schema

The config file is multi-purpose by design - a unified "services" concept that covers LLMs, TTS, browsing, search, and future capabilities.

# bigagi.yaml
version: 1

# --- Service Declarations ---
# Each service is a configured endpoint that provides capabilities.
# The config key becomes the stable service UID (prefixed with "srv.").

services:

  # --- LLM Services ---

  openai:
    type: llm                          # service type: llm | tts | browsing | search
    vendor: openai                     # maps to ModelVendorId / ISpeexVendor etc.
    label: "OpenAI"                    # display name (optional, defaults to vendor name)
    auth:
      apiKey: "${OPENAI_API_KEY}"      # env var interpolation
      apiOrg: "${OPENAI_API_ORG_ID}"
    endpoint:
      host: "${OPENAI_API_HOST}"       # optional: custom host
    helicone:
      apiKey: "${HELICONE_API_KEY}"    # optional: proxy integration
    models:                            # optional: model filtering
      filter: allowlist                # allowlist | denylist
      patterns:                        # glob patterns matched against model ref IDs
        - "gpt-4o*"
        - "o3*"
        - "o4*"
    locked: true                       # default true: client cannot modify credentials
    monotonic: 1                       # bump to force model re-scan for this service

  openai-dev:
    type: llm
    vendor: openai
    label: "OpenAI (Dev)"
    auth:
      apiKey: "${OPENAI_DEV_API_KEY}"
    models:
      filter: denylist
      patterns:
        - "dall-e*"                    # exclude image models
    monotonic: 1

  gemini:
    type: llm
    vendor: gemini
    label: "Gemini"
    auth:
      apiKey: "${GEMINI_API_KEY}"
    endpoint:
      host: "${GEMINI_API_HOST}"       # the original issue request
    vendorConfig:                      # vendor-specific config (maps to access schema extras)
      minSafetyLevel: BLOCK_NONE
    monotonic: 1

  ollama-local:
    type: llm
    vendor: ollama
    label: "Ollama (GPU Server)"
    endpoint:
      host: "http://gpu-server:11434"
    locked: false                      # user CAN adjust this (local service)
    monotonic: 1

  anthropic:
    type: llm
    vendor: anthropic
    label: "Anthropic"
    auth:
      apiKey: "${ANTHROPIC_API_KEY}"
    endpoint:
      host: "${ANTHROPIC_API_HOST}"
    monotonic: 1

  azure:
    type: llm
    vendor: azure
    label: "Azure OpenAI"
    auth:
      apiKey: "${AZURE_OPENAI_API_KEY}"
    endpoint:
      host: "${AZURE_OPENAI_API_ENDPOINT}"
    vendorConfig:
      disableV1: false
      apiVersion: "2025-04-01-preview"
      deploymentsApiVersion: "2023-03-15-preview"
    monotonic: 1

  bedrock:
    type: llm
    vendor: bedrock
    label: "AWS Bedrock"
    auth:
      bearerToken: "${BEDROCK_BEARER_TOKEN}"
      accessKeyId: "${BEDROCK_ACCESS_KEY_ID}"
      secretAccessKey: "${BEDROCK_SECRET_ACCESS_KEY}"
      sessionToken: "${BEDROCK_SESSION_TOKEN}"
    vendorConfig:
      region: "${BEDROCK_REGION}"
    monotonic: 1

  # --- TTS Services ---

  elevenlabs:
    type: tts
    vendor: elevenlabs
    label: "ElevenLabs TTS"
    auth:
      apiKey: "${ELEVENLABS_API_KEY}"
    endpoint:
      host: "${ELEVENLABS_API_HOST}"
    vendorConfig:
      voiceId: "${ELEVENLABS_VOICE_ID}"
    monotonic: 1

  # --- Browsing Services ---

  browsing:
    type: browsing
    vendor: puppeteer
    label: "Web Browsing"
    endpoint:
      host: "${PUPPETEER_WSS_ENDPOINT}"
    monotonic: 1

  # --- Search Services ---

  google-search:
    type: search
    vendor: google-cse
    label: "Google Search"
    auth:
      apiKey: "${GOOGLE_CLOUD_API_KEY}"
    vendorConfig:
      cseId: "${GOOGLE_CSE_ID}"
    monotonic: 1


# --- Global Settings ---

settings:
  httpBasicAuth:
    username: "${HTTP_BASIC_AUTH_USERNAME}"
    password: "${HTTP_BASIC_AUTH_PASSWORD}"
  aixStrictParsing: false

Key design decisions in the schema:

1. Uniform type field - every service declares its category (llm, tts, browsing, search). Extensible without schema changes.

2. Uniform auth/endpoint structure - all services use the same top-level shape. Vendor-specific fields go into vendorConfig.

3. ${VAR} interpolation - shell-like, consistent with Docker Compose and LibreChat (not LiteLLM's os.environ/). Secrets never appear in the YAML file itself.

4. monotonic field - deployer bumps this number to force a model re-scan for that specific service.

5. Config key = stable service UID - openai-dev becomes service UID srv.openai-dev. Deterministic, deployer-controlled, stable across all clients.

6. locked: true default - server-declared services are locked by default. locked: false for local services where users may need to adjust the host.

3. Server-to-Client Protocol

Replace the flat hasLlmX booleans with a structured ServerDeclaredService protocol.

interface ServerDeclaredService {
  // Identity (derived from config)
  uid: string;               // "srv.openai", "srv.openai-dev", "srv.gemini"
  type: ServiceType;         // "llm" | "tts" | "browsing" | "search"
  vendorId: string;          // ModelVendorId for LLMs, DSpeexVendorType for TTS, etc.
  label: string;             // display name

  // Capabilities (what the server configured - NO secrets)
  hasAuth: boolean;          // server has credentials configured
  hasEndpoint: boolean;      // server has a custom endpoint configured

  // Model filtering (server-enforced)
  modelFilter?: {
    mode: 'allowlist' | 'denylist';
    patterns: string[];
  };

  // UI behavior
  locked: boolean;           // client cannot modify credentials

  // Refresh control
  monotonic: number;         // per-service version for targeted refresh
}

type ServiceType = 'llm' | 'tts' | 'browsing' | 'search';

interface BackendCapabilities {
  // NEW: structured service declarations
  serverServices: ServerDeclaredService[];

  // KEPT: non-service capabilities (until migrated)
  hasDB: boolean;

  // KEPT: global reconfig hash
  hashLlmReconfig: string;

  // KEPT
  build?: { gitSha?: string; pkgVersion?: string; timestamp?: string };
}

Key: The client receives UIDs and metadata, never secrets. The server resolves credentials internally when the client references a service by UID.

4. Per-Service Monotonic + Global Hash

function generateConfigHash(
  serverServices: ServerDeclaredService[],
  globalMonotonic: number,  // Release.Monotonics.Aix
): string {
  const inputs = [
    globalMonotonic.toString(),
    Release.TenantSlug.toString(),
    // per-service: uid + monotonic
    ...serverServices.map(s => `${s.uid}:${s.monotonic}`).sort(),
  ];
  return sdbmHash(inputs.join(';'));
}

Client-side refresh logic:

for (const serverService of serverServices) {
  const existing = findServiceByUid(serverService.uid);
  if (!existing) {
    createAndFetchService(serverService);          // new service
  } else if (existing.monotonic !== serverService.monotonic) {
    refetchModelsForService(existing);             // monotonic bumped
  }
  // else: no change, skip
}

This avoids re-scanning all vendors when only one changes.

5. Security/Authorization Framework

The design establishes a UID-based authorization boundary where the client never possesses or transmits secrets.

┌─────────────────────────────────────────────────────────────────┐
│                        CLIENT                                    │
│                                                                  │
│  Knows: service UIDs, model IDs, vendor metadata                │
│  Does NOT know: API keys, bearer tokens, host overrides          │
│                                                                  │
│  Operations:                                                     │
│  - "List models for srv.openai"  (by UID)                       │
│  - "Generate with srv.openai@gpt-4o"  (by model ID)            │
│  - "Speak text via srv.elevenlabs"  (by UID)                   │
│  - "Browse url via srv.browsing"  (by UID)                     │
│                                                                  │
│  Cannot:                                                         │
│  - Modify locked service credentials                             │
│  - Use a service UID that wasn't declared by the server          │
│  - Access models filtered out by server-side allowlist/denylist  │
│  - Forge a service UID to bypass model filtering                 │
└──────────────────────────────────┬──────────────────────────────┘
                                   │
                                   │ tRPC (service UID + model ref)
                                   │
┌──────────────────────────────────▼──────────────────────────────┐
│                        SERVER                                    │
│                                                                  │
│  Config loader (YAML + env var interpolation)                   │
│       │                                                          │
│       ▼                                                          │
│  ServiceRegistry (in-memory, immutable after load)              │
│  ┌──────────────────────────────────────────┐                   │
│  │ srv.openai     -> { auth, endpoint, ... }│                   │
│  │ srv.gemini     -> { auth, endpoint, ... }│                   │
│  │ srv.elevenlabs -> { auth, endpoint, ... }│                   │
│  │ srv.browsing   -> { auth, endpoint, ... }│                   │
│  └──────────────────────────────────────────┘                   │
│       │                                                          │
│       ▼                                                          │
│  Authorization checks:                                           │
│  1. Is this UID a registered server service?                    │
│  2. Is the requested model allowed by the service's filter?     │
│  3. Resolve credentials from registry (never from client)       │
│  4. Dispatch to vendor API                                       │
└─────────────────────────────────────────────────────────────────┘

Authorization Flow

async function handleChatGenerate(request: {
  serviceUid: string;       // "srv.openai" - from client
  modelRef: string;         // "gpt-4o" - from client
  messages: AixMessage[];
}) {
  // 1. Resolve service from registry
  const service = serviceRegistry.get(request.serviceUid);
  if (!service)
    throw new TRPCError({ code: 'NOT_FOUND', message: 'Unknown service' });

  // 2. Enforce model filter
  if (service.modelFilter) {
    const allowed = service.modelFilter.mode === 'allowlist'
      ? matchesAny(request.modelRef, service.modelFilter.patterns)
      : !matchesAny(request.modelRef, service.modelFilter.patterns);
    if (!allowed)
      throw new TRPCError({ code: 'FORBIDDEN', message: 'Model not allowed' });
  }

  // 3. Build access object from server-side credentials (NOT from client)
  const access = service.buildAccessObject();

  // 4. Dispatch to vendor
  return dispatchToVendor(service.vendorId, access, request.modelRef, request.messages);
}

Coexistence: Server + Client Services

Aspect	Server Service (srv.*)	Client Service
Service UID	srv.{configKey}	{vendorId} or {vendorId}.{n}
Credentials	Server-side only	Client-side (localStorage)
Model ID format	srv.openai@gpt-4o	openai@gpt-4o
Request flow	Client sends UID -> Server resolves creds	Client sends full access object
Deletable	No (reappears on refresh)	Yes
Editable	Only if locked: false	Always
Model filter	Server-enforced	None

6. Config File Loader Design

// src/server/config/config.loader.ts

interface BigAGIConfig {
  version: number;
  services: Record<string, RawServiceConfig>;
  settings?: RawSettingsConfig;
}

interface RawServiceConfig {
  type: ServiceType;
  vendor: string;
  label?: string;
  auth?: Record<string, string>;
  endpoint?: { host?: string };
  vendorConfig?: Record<string, unknown>;
  models?: { filter: 'allowlist' | 'denylist'; patterns: string[] };
  locked?: boolean;                   // default: true
  monotonic?: number;                 // default: 1
}

function loadConfig(): BigAGIConfig | null {
  const configPath = process.env.BIGAGI_CONFIG
    || findConfigFile(['bigagi.yaml', 'bigagi.yml', 'bigagi.json']);
  if (!configPath) return null;

  const raw = readFileSync(configPath, 'utf-8');
  const parsed = configPath.endsWith('.json')
    ? JSON.parse(raw)
    : yamlParse(raw);
  const interpolated = deepInterpolateEnvVars(parsed);
  return bigagiConfigSchema.parse(interpolated); // Zod fail-fast (LibreChat pattern)
}

function deepInterpolateEnvVars(obj: unknown): unknown {
  if (typeof obj === 'string') {
    return obj.replace(/\$\{([^}]+)\}/g, (_, varName) => {
      const value = process.env[varName];
      if (value === undefined)
        console.warn(`[config] Environment variable ${varName} is not set`);
      return value ?? '';
    });
  }
  if (Array.isArray(obj)) return obj.map(deepInterpolateEnvVars);
  if (obj && typeof obj === 'object')
    return Object.fromEntries(
      Object.entries(obj).map(([k, v]) => [k, deepInterpolateEnvVars(v)])
    );
  return obj;
}

7. Features Adopted from Competitor Research

Feature	Source	Adopted?	Rationale
${VAR} interpolation	LibreChat	Yes	Universal, Docker Compose compatible
Zod fail-fast validation	LibreChat	Yes	Already using Zod; catches misconfig at startup
models.fetch + filter	LibreChat	Yes	Auto-fetch + allowlist/denylist patterns
Per-service monotonic	Novel	Yes	Targeted refresh without full re-scan
Version field	LibreChat	Yes	Forward-compatible schema evolution
dropParams	LibreChat	Deferred	Currently handled via interfaces hotfix flags
Model list DSL (-all,+gpt-4o)	LobeChat	Deferred	Elegant for env vars, but YAML patterns are richer
Load balancing strategies	LiteLLM	Deferred	Multi-key rotation already exists via key1,key2
PersistentConfig (DB > env)	Open WebUI	Noted	Relevant when DB tier matures (Pro)
Virtual key vault	Portkey	Noted	Future Pro feature for managed key rotation
Hierarchical RBAC	LiteLLM	Noted	Future extension when user auth exists
Semantic caching	Portkey	Watching	Cost reduction for common queries
Model "wrapping" (bind tools/prompts)	Open WebUI	Watching	Could enhance Beam/personas

8. Impact on Existing Access Files

Server-declared services bypass the existing access.oaiKey || env.VENDOR_KEY || '' pattern entirely. For srv.* services, the server resolves credentials from the config registry. Existing access files remain unchanged for client-added services.

srv.openai@gpt-4o:
  -> Server looks up srv.openai in ServiceRegistry
  -> Builds access object from config
  -> Calls openAIAccess(serverAccess, "gpt-4o", apiPath)

openai@gpt-4o (client service):
  -> Client sends full access object (existing behavior)
  -> Server calls openAIAccess(clientAccess, "gpt-4o", apiPath)

9. UI Design for Server vs Client Services

┌─ Models Setup ────────────────────────────────────────┐
│                                                        │
│  SERVER-CONFIGURED                                     │
│  ┌──────────────────────────────────────────────────┐  │
│  │ 🔒 OpenAI                          [Refresh]    │  │
│  │    Models: 12 available (filtered: gpt-4o*, o3*) │  │
│  ├──────────────────────────────────────────────────┤  │
│  │ 🔒 Anthropic                       [Refresh]    │  │
│  │    Models: 8 available                           │  │
│  ├──────────────────────────────────────────────────┤  │
│  │ 🔓 Ollama (GPU Server)             [Refresh]    │  │
│  │    Host: http://gpu-server:11434   [Edit]        │  │
│  │    Models: 3 available                           │  │
│  └──────────────────────────────────────────────────┘  │
│                                                        │
│  YOUR SERVICES                                         │
│  ┌──────────────────────────────────────────────────┐  │
│  │    Groq (Personal)                 [Delete]      │  │
│  └──────────────────────────────────────────────────┘  │
│                                                        │
│  [+ Add Service]                                       │
│                                                        │
└────────────────────────────────────────────────────────┘

10. Implementation Phases

Phase 0: Quick win (this PR or next)
Add missing env vars for the vendors identified in the issue. Mechanical, no architecture change. Unblocks Docker users.

Phase 1: Config loader + Service Registry

• Add yaml dependency
• Implement config.loader.ts with ${VAR} interpolation + Zod validation
• Implement ServiceRegistry (in-memory, immutable)
• New BackendCapabilities.serverServices with ServerDeclaredService[]
• Backward-compat: derive hasLlmX from serverServices (or generate from env vars if no config file)

Phase 2: UID-based authorization

• Update AIX request handler to accept service UID
• Server-side credential resolution from registry
• Model filter enforcement (list + request)
• Update AixAPI_Access to support UID-based access

Phase 3: Model ID migration

• Switch from - to @ separator for new service IDs
• Update _createDLLMFromModelDescription and getDLLMCloneId
• Client-side parser for both old and new format

Phase 4: UI separation

• Server vs client service sections
• Locked service UI (hidden creds, no delete)
• Model filter badges
• Extend to TTS/Browsing/Search service types

---

Open Questions for Decision

1. Config file search order: ./bigagi.yaml > ./bigagi.yml > ./bigagi.json? Or require BIGAGI_CONFIG env var?

2. Fail behavior: Fail-fast on invalid config (LibreChat pattern) or warn-and-skip invalid entries?

3. @ separator: Any concern with @ in URL fragments or existing data? Alternative: ~ or | (but less intuitive).

4. Multi-key support in YAML: Keep comma-separated "key1,key2,key3" pattern? Or add explicit apiKeys: [key1, key2, key3] array syntax?

5. Config reload: Hot-reload on file change (via file watcher), or require server restart?