Systems Engineer | Maintainer of Bifrost | Engineering @ Maxim AI
I build high-performance backend systems and AI infrastructure.
Currently building Bifrost, the fastest AI gateway designed for scale:
- 10K+ RPS on t3.medium
- <15µs internal overhead
- 40x faster than LiteLLM
- Production deployments handling billions of tokens daily
Languages:
- Primary: Go, TypeScript
- Also worked with: Python, C/C++, Java, R
High-performance AI gateway built in Go, architected and developed from the ground up. Some differentiating features I designed and implemented:
-
Core Architecture
End-to-end execution engine built for zero runtime allocations and sustained high-throughput production workloads. -
Adaptive Load Balancing Engine
A multi-level routing algorithm that continuously evaluates real-time metrics such as latency, success rates, utilization, and cost to dynamically distribute traffic across providers and API keys, ensuring performance, resilience, and fair utilization. -
Dynamic Routing Rules Framework
Expression-based routing powered by CEL, evaluated at request time with scoped precedence (virtual key → team → customer → global), enabling advanced conditional routing based on request context, headers, and system state. -
Governance and Virtual Keys
Policy and access-control layer managing budgets, rate limits, model access, and fine-grained provider permissions per consumer. -
MCP Gateway
Scalable Model Context Protocol gateway exposing tools securely through filtered virtual key permissions and unified provider orchestration.
Distributed collaboration platform built and operated end-to-end: `
- 6,000+ active users
- 100K+ requests in 7 days during peak usage
- Official hosting platform for major university events with 5,000+ participants
- Architected 11 microservices deployed via Docker, Nginx, and GCP
- Dedicated ML microservice for personalized recommendations, moderation, and automated code reviews
- Integrated Stripe for payments, Temporal for workflow orchestration, and Celery for background processing