-
Notifications
You must be signed in to change notification settings - Fork 2
feat(llm): expose warmup_queries as TOML config field in [llm.router.bandit] #2543
Description
Problem
The warmup_queries field appears in the TOML config ([llm.router.bandit]) in user documentation and configs but is silently ignored — it is not part of BanditConfig (the TOML-facing struct).
The actual warmup threshold is hardcoded in bootstrap as 10 * num_providers and cannot be overridden.
Impact: With 2 providers, warmup requires 20 queries. Operators cannot reduce this for testing or faster cold-start (e.g., warmup_queries = 1 is a common debugging need). The silent ignore is misleading.
Expected
warmup_queries in [llm.router.bandit] TOML section should override the default 10 * num_providers warmup threshold.
Actual
BanditConfig has no warmup_queries field. The value is silently ignored. Warmup is always 10 * num_providers.
Discovered
CI-358 (2026-03-31) — bandit routing test. State had 12 updates with 2 providers: warmup threshold = 20, so LinUCB never activated. Tested workaround: drop to 1 provider (warmup = 10, 12 > 10 → LinUCB active).
Fix
Add warmup_queries: Option<u64> to BanditConfig with #[serde(default)]. In bootstrap, use it as override when set: warmup_queries = cfg.warmup_queries.unwrap_or(10 * num_providers).