Multi-provider adapter: OpenAI / Claude / DeepSeek unified interface

[zjy4fun]

Goal

Abstract the provider layer so the system can route requests to multiple LLM providers through a unified interface. This turns the project from "OpenAI wrapper" into a lightweight AI Gateway.

Current State

provider.ts hardcodes OpenAI SDK. Only OpenAI-compatible APIs work.

Tasks

• Define LLMProvider interface: chatComplete(params) and chatStream(params) with unified input/output types
• Implement OpenAIProvider — wrap current logic
• Implement ClaudeProvider — Anthropic SDK adapter (messages API, different streaming format)
• Implement DeepSeekProvider — OpenAI-compatible but with provider-specific quirks
• Provider registry: register providers by name, lookup at runtime
• Config: server/.env defines available providers with their API keys and base URLs
• Normalize responses: map each provider's response to a common ChatResponse / StreamChunk type
• Handle provider-specific differences:
  • Token counting (different tokenizers)
  • Max context lengths
  • Tool calling format differences (OpenAI tools vs Claude tool_use)
  • Streaming chunk format normalization

Smart Router (extends #6)

• Cost-based routing: simple questions → cheapest model, complex → strongest
• Latency-based routing: pick the fastest responding provider
• User preference: allow user to select preferred provider/model
• Fallback chain: configurable per-provider (e.g. OpenAI → DeepSeek → Claude)

Why

This is the architecture behind services like OpenRouter, LiteLLM, and AI Gateway services. Building it yourself gives you deep understanding of how LLM gateways work, API differences between providers, and the adapter pattern in practice.

Learning Points

• Adapter pattern: wrapping incompatible interfaces behind a common contract
• Provider-specific streaming formats and how to normalize them
• Cost optimization strategies for multi-model systems
• Configuration-driven architecture (add new provider without code changes)