Using LiteLLM router (with fallback and retry/cooldown) in an Agent

[NateMoneyLion]

Re-echoing issue 813 to allow for setting up fallback models for a given Agent instance.

[seratch]

Thanks for raising this. A LiteLLM Router instance cannot be passed directly as Agent.model, because that field expects an Agents SDK Model implementation. However, the fallback use case does not require an SDK change.

LitellmModel forwards ModelSettings.extra_args to litellm.acompletion(), so request-level retries and fallbacks can be configured like this:

    ModelSettings(
        extra_args={
            "num_retries": 2,
            "fallbacks": ["provider/backup-model"],
        }
    )

If you need the Router's long-lived deployment state, load balancing, failure counters, and cooldown behavior across requests, the recommended boundary is the LiteLLM Proxy. Configure the Router there and point an OpenAIChatCompletionsModel, backed by AsyncOpenAI(base_url=...), at its OpenAI-compatible endpoint.

Since these existing paths cover the underlying outcome, we do not plan to add a Router-specific model contract to the SDK. I am closing this as answered.