Skip to content

Commit 273ebee

Browse files
Remove OllamaModel in favor of usage with OpenAIModel (#805)
1 parent bb4c828 commit 273ebee

File tree

12 files changed

+85
-331
lines changed

12 files changed

+85
-331
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ We built PydanticAI with one simple aim: to bring that FastAPI feeling to GenAI
3737
Built by the team behind [Pydantic](https://docs.pydantic.dev/latest/) (the validation layer of the OpenAI SDK, the Anthropic SDK, LangChain, LlamaIndex, AutoGPT, Transformers, CrewAI, Instructor and many more).
3838

3939
* __Model-agnostic__
40-
Supports OpenAI, Anthropic, Gemini, Ollama, Groq, and Mistral, and there is a simple interface to implement support for [other models](https://ai.pydantic.dev/models/).
40+
Supports OpenAI, Anthropic, Gemini, Deepseek, Ollama, Groq, Cohere, and Mistral, and there is a simple interface to implement support for [other models](https://ai.pydantic.dev/models/).
4141

4242
* __Pydantic Logfire Integration__
4343
Seamlessly [integrates](https://ai.pydantic.dev/logfire/) with [Pydantic Logfire](https://pydantic.dev/logfire) for real-time debugging, performance monitoring, and behavior tracking of your LLM-powered applications.

docs/api/models/ollama.md

Lines changed: 0 additions & 75 deletions
This file was deleted.

docs/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ We built PydanticAI with one simple aim: to bring that FastAPI feeling to GenAI
1717
Built by the team behind [Pydantic](https://docs.pydantic.dev/latest/) (the validation layer of the OpenAI SDK, the Anthropic SDK, LangChain, LlamaIndex, AutoGPT, Transformers, CrewAI, Instructor and many more).
1818

1919
:fontawesome-solid-shapes:{ .md .middle .shapes-orange }&nbsp;<strong class="vertical-middle">Model-agnostic</strong><br>
20-
Supports OpenAI, Anthropic, Gemini, Ollama, Groq, and Mistral, and there is a simple interface to implement support for [other models](models.md).
20+
Supports OpenAI, Anthropic, Gemini, Deepseek, Ollama, Groq, Cohere, and Mistral, and there is a simple interface to implement support for [other models](models.md).
2121

2222
:logfire-logo:{ .md .middle }&nbsp;<strong class="vertical-middle">Pydantic Logfire Integration</strong><br>
2323
Seamlessly [integrates](logfire.md) with [Pydantic Logfire](https://pydantic.dev/logfire) for real-time debugging, performance monitoring, and behavior tracking of your LLM-powered applications.

docs/models.md

Lines changed: 76 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,11 @@ PydanticAI is Model-agnostic and has built in support for the following model pr
44
* [Anthropic](#anthropic)
55
* Gemini via two different APIs: [Generative Language API](#gemini) and [VertexAI API](#gemini-via-vertexai)
66
* [Ollama](#ollama)
7+
* [Deepseek](#deepseek)
78
* [Groq](#groq)
89
* [Mistral](#mistral)
910

10-
See [OpenAI-compatible models](#openai-compatible-models) for more examples on how to use models such as [OpenRouter](#openrouter), [Grok (xAI)](#grok-xai) and [DeepSeek](#deepseek) that support the OpenAI SDK.
11+
See [OpenAI-compatible models](#openai-compatible-models) for more examples on how to use models such as [OpenRouter](#openrouter), and [Grok (xAI)](#grok-xai) that support the OpenAI SDK.
1112

1213
You can also [add support for other models](#implementing-custom-models).
1314

@@ -304,26 +305,6 @@ agent = Agent(model)
304305

305306
[`VertexAiRegion`][pydantic_ai.models.vertexai.VertexAiRegion] contains a list of available regions.
306307

307-
## Ollama
308-
309-
### Install
310-
311-
To use [`OllamaModel`][pydantic_ai.models.ollama.OllamaModel], you need to either install [`pydantic-ai`](install.md), or install [`pydantic-ai-slim`](install.md#slim-install) with the `openai` optional group:
312-
313-
```bash
314-
pip/uv-add 'pydantic-ai-slim[openai]'
315-
```
316-
317-
**This is because internally, `OllamaModel` uses the OpenAI API.**
318-
319-
### Configuration
320-
321-
To use [Ollama](https://ollama.com/), you must first download the Ollama client, and then download a model using the [Ollama model library](https://ollama.com/library).
322-
323-
You must also ensure the Ollama server is running when trying to make requests to it. For more information, please see the [Ollama documentation](https://github.com/ollama/ollama/tree/main/docs).
324-
325-
For detailed setup and example, please see the [Ollama setup documentation](https://github.com/pydantic/pydantic-ai/blob/main/docs/api/models/ollama.md).
326-
327308
## Groq
328309

329310
### Install
@@ -456,6 +437,80 @@ model = OpenAIModel(
456437
...
457438
```
458439

440+
### Ollama
441+
442+
To use [Ollama](https://ollama.com/), you must first download the Ollama client, and then download a model using the [Ollama model library](https://ollama.com/library).
443+
444+
You must also ensure the Ollama server is running when trying to make requests to it. For more information, please see the [Ollama documentation](https://github.com/ollama/ollama/tree/main/docs).
445+
446+
#### Example local usage
447+
448+
With `ollama` installed, you can run the server with the model you want to use:
449+
450+
```bash {title="terminal-run-ollama"}
451+
ollama run llama3.2
452+
```
453+
(this will pull the `llama3.2` model if you don't already have it downloaded)
454+
455+
Then run your code, here's a minimal example:
456+
457+
```python {title="ollama_example.py"}
458+
from pydantic import BaseModel
459+
460+
from pydantic_ai import Agent
461+
from pydantic_ai.models.openai import OpenAIModel
462+
463+
464+
class CityLocation(BaseModel):
465+
city: str
466+
country: str
467+
468+
469+
ollama_model = OpenAIModel(model_name='llama3.2', base_url='http://localhost:11434/v1')
470+
agent = Agent(ollama_model, result_type=CityLocation)
471+
472+
result = agent.run_sync('Where were the olympics held in 2012?')
473+
print(result.data)
474+
#> city='London' country='United Kingdom'
475+
print(result.usage())
476+
"""
477+
Usage(requests=1, request_tokens=57, response_tokens=8, total_tokens=65, details=None)
478+
"""
479+
```
480+
481+
#### Example using a remote server
482+
483+
```python {title="ollama_example_with_remote_server.py"}
484+
from pydantic import BaseModel
485+
486+
from pydantic_ai import Agent
487+
from pydantic_ai.models.openai import OpenAIModel
488+
489+
ollama_model = OpenAIModel(
490+
model_name='qwen2.5-coder:7b', # (1)!
491+
base_url='http://192.168.1.74:11434/v1', # (2)!
492+
)
493+
494+
495+
class CityLocation(BaseModel):
496+
city: str
497+
country: str
498+
499+
500+
agent = Agent(model=ollama_model, result_type=CityLocation)
501+
502+
result = agent.run_sync('Where were the olympics held in 2012?')
503+
print(result.data)
504+
#> city='London' country='United Kingdom'
505+
print(result.usage())
506+
"""
507+
Usage(requests=1, request_tokens=57, response_tokens=8, total_tokens=65, details=None)
508+
"""
509+
```
510+
511+
1. The name of the model running on the remote server
512+
2. The url of the remote server
513+
459514
### OpenRouter
460515

461516
To use [OpenRouter](https://openrouter.ai), first create an API key at [openrouter.ai/keys](https://openrouter.ai/keys).

mkdocs.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,6 @@ nav:
5454
- api/models/vertexai.md
5555
- api/models/groq.md
5656
- api/models/mistral.md
57-
- api/models/ollama.md
5857
- api/models/test.md
5958
- api/models/function.md
6059
- api/pydantic_graph/graph.md

pydantic_ai_slim/pydantic_ai/models/__init__.py

Lines changed: 2 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,10 @@
1212
from dataclasses import dataclass, field
1313
from datetime import datetime
1414
from functools import cache
15-
from typing import TYPE_CHECKING, Literal
15+
from typing import TYPE_CHECKING
1616

1717
import httpx
18+
from typing_extensions import Literal
1819

1920
from .._parts_manager import ModelResponsePartsManager
2021
from ..exceptions import UserError
@@ -107,25 +108,6 @@
107108
'o1-mini-2024-09-12',
108109
'o1-preview',
109110
'o1-preview-2024-09-12',
110-
'ollama:codellama',
111-
'ollama:deepseek-r1',
112-
'ollama:gemma',
113-
'ollama:gemma2',
114-
'ollama:llama3',
115-
'ollama:llama3.1',
116-
'ollama:llama3.2',
117-
'ollama:llama3.2-vision',
118-
'ollama:llama3.3',
119-
'ollama:mistral',
120-
'ollama:mistral-nemo',
121-
'ollama:mixtral',
122-
'ollama:phi3',
123-
'ollama:phi4',
124-
'ollama:qwen',
125-
'ollama:qwen2',
126-
'ollama:qwen2.5',
127-
'ollama:qwq',
128-
'ollama:starcoder2',
129111
'openai:chatgpt-4o-latest',
130112
'openai:gpt-3.5-turbo',
131113
'openai:gpt-3.5-turbo-0125',
@@ -356,10 +338,6 @@ def infer_model(model: Model | KnownModelName) -> Model:
356338
from .mistral import MistralModel
357339

358340
return MistralModel(model[8:])
359-
elif model.startswith('ollama:'):
360-
from .ollama import OllamaModel
361-
362-
return OllamaModel(model[7:])
363341
elif model.startswith('anthropic'):
364342
from .anthropic import AnthropicModel
365343

pydantic_ai_slim/pydantic_ai/models/ollama.py

Lines changed: 0 additions & 123 deletions
This file was deleted.

0 commit comments

Comments
 (0)