Commit e7b02fa
feat: Production-grade provider manager with resilience patterns
Enhanced provider manager with:
- Circuit breaker pattern: Automatic failover when providers fail
(threshold: 5 failures, 1min recovery)
- LRU caching: Response caching with TTL and size limits (1000 max)
- Rate limiting: Per-provider token bucket (60/min cloud, 1000/min local)
- Health checks: Periodic provider availability monitoring
- Latency percentiles: p50/p95/p99 tracking for SLA monitoring
- Model-specific pricing: Accurate cost tracking per model variant
- Token extraction: Real usage tracking from API responses
Tests expanded from 10 to 40+ covering all new features.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1 parent 90d4c03 commit e7b02fa
File tree
2 files changed
+991
-159
lines changed- lux
- lib/lux/llm
- test/lux/llm
2 files changed
+991
-159
lines changed
0 commit comments