Adding LiteLLM as model wrap just like how google-adk does it.

[frederikhendrix]

Description

LiteLLM seems to have all big providers and even things like Ollama and LMstudio. If it is possible it would probably be best for everybody if it is possible to use LiteLLM as a providor for the tool calls. This way Pydantic-ai can stop focussing on adding new AI's to the list and just focus on being an agentic framework.

References

No response

[samuelcolvin]

I think we can add LiteLLM as a model, but it's worth noting that we already support all big provider, Ollama and LMStudio, see https://ai.pydantic.dev/models/openai/#openai-compatible-models.

[DouweM]

LiteLLM has an OpenAI-compatible API, so it should be easy enough to add a LiteLLMProvider that can be used with OpenAIModel. That'd be similar to OpenRouter, Together, Fireworks etc.

Together with #1835, we can also have automatically pick the right model profile (which configures things like the JSON schema transformer) based on the model name prefix.

[NorthIsUp]

+1 to litellm support.

Litellm's value is beyond its model abstraction layer. It also offers budget aware routing between providers (e.g. spreading budget across vertex/azure/bedrock). It offers virtual api keys so you can track usage across users/services. And a ton of other useful features that make our lives easier as an early stage startup.

🙏

[tajkirkpatrick]

Support for LiteLLM would be really appreciated!

[lvicentesanchez]

It would certainly make things simpler.

[sergioave]

It would be really interesting to have a provider for litellm.

[samuelcolvin]

PR welcome to implement this!

[mochow13]

Hello all! Inspired by langchain-litellm, I created pydantic-ai-litellm to integrate LiteLLM models with pydantic-ai. The very first release was done just a short while ago.

Repository: https://github.com/mochow13/pydantic-ai-litellm
PyPI: https://pypi.org/project/pydantic-ai-litellm/

Please feel free to raise issues and PRs.

[DouweM]

@mochow13 Nice work! Would you be interested in contributing this directly to Pydantic AI?

[timfdev]

@mochow13 Yes would like to see this too in pydantic-ai directly🙏

[mochow13]

Hi @timfdev and @DouweM, sure I think I can raise an MR. But will take some time due to holiday season.

And thanks for your feedback!