Skip to content

Commit a574c05

Browse files
committed
Document the Outlines model
1 parent eb1e443 commit a574c05

File tree

7 files changed

+88
-3
lines changed

7 files changed

+88
-3
lines changed

docs/api/models/outlines.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# `pydantic_ai.models.outlines`
2+
3+
## Setup
4+
5+
For details on how to set up this model, see [model configuration for Outlines](../../models/outlines.md).
6+
7+
::: pydantic_ai.models.outlines

docs/builtin-tools.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ making it ideal for queries that require up-to-date data.
3232
| Mistral || Not supported |
3333
| Cohere || Not supported |
3434
| HuggingFace || Not supported |
35+
| Outlines || Not supported |
3536

3637
!!! note "Groq Support"
3738
To use web search capabilities with Groq, you need to use the [compound models](https://console.groq.com/docs/compound).
@@ -107,6 +108,7 @@ in a secure environment, making it perfect for computational tasks, data analysi
107108
| Mistral ||
108109
| Cohere ||
109110
| HuggingFace ||
111+
| Outlines ||
110112

111113
### Usage
112114

@@ -136,6 +138,7 @@ allowing it to pull up-to-date information from the web.
136138
| Mistral ||
137139
| Cohere ||
138140
| HuggingFace ||
141+
| Outlines ||
139142

140143
### Usage
141144

docs/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ We built Pydantic AI with one simple aim: to bring that FastAPI feeling to GenAI
2424
Built by the team behind [Pydantic Validation](https://docs.pydantic.dev/latest/) (the validation layer of the OpenAI SDK, the Anthropic SDK, LangChain, LlamaIndex, AutoGPT, Transformers, CrewAI, Instructor and many more).
2525

2626
- **Model-agnostic**:
27-
Supports OpenAI, Anthropic, Gemini, Deepseek, Ollama, Groq, Cohere, and Mistral, and there is a simple interface to implement support for [other models](models/overview.md).
27+
Supports OpenAI, Anthropic, Gemini, Deepseek, Ollama, Groq, Cohere, Mistral, and Outlines, and there is a simple interface to implement support for [other models](models/overview.md).
2828

2929
- **Pydantic Logfire Integration**:
3030
Seamlessly [integrates](logfire.md) with [Pydantic Logfire](https://pydantic.dev/logfire) for real-time debugging, performance monitoring, and behavior tracking of your LLM-powered applications.

docs/install.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ pip/uv-add "pydantic-ai-slim[openai]"
5151
* `cohere` - installs `cohere` [PyPI ↗](https://pypi.org/project/cohere){:target="_blank"}
5252
* `bedrock` - installs `boto3` [PyPI ↗](https://pypi.org/project/boto3){:target="_blank"}
5353
* `huggingface` - installs `huggingface-hub[inference]` [PyPI ↗](https://pypi.org/project/huggingface-hub){:target="_blank"}
54+
* `outlines` - installs `outlines` [PyPI ↗](https://pypi.org/project/outlines){:target="_blank"}
5455
* `duckduckgo` - installs `ddgs` [PyPI ↗](https://pypi.org/project/ddgs){:target="_blank"}
5556
* `tavily` - installs `tavily-python` [PyPI ↗](https://pypi.org/project/tavily-python){:target="_blank"}
5657
* `cli` - installs `rich` [PyPI ↗](https://pypi.org/project/rich){:target="_blank"}, `prompt-toolkit` [PyPI ↗](https://pypi.org/project/prompt-toolkit){:target="_blank"}, and `argcomplete` [PyPI ↗](https://pypi.org/project/argcomplete){:target="_blank"}

docs/models/outlines.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# Outlines
2+
3+
## Install
4+
5+
To use `OutlinesModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `outlines` optional group:
6+
7+
```bash
8+
pip/uv-add "pydantic-ai-slim[outlines]"
9+
```
10+
11+
As Outlines is a library allowing you to run model from various different providers, it does not include the necessary dependencies for any model, but instead requires you to install the optional group for the provider you want to use. For instance:
12+
13+
```bash
14+
pip/uv-add "outlines[transformers]"
15+
```
16+
17+
Or
18+
19+
```bash
20+
pip/uv-add "outlines[mlxlm]"
21+
```
22+
23+
## Model Initialization
24+
25+
As Outlines is not an inference provider, but instead a library allowing you to run bith local and API-based models, instantiating the model is a bit different from the other models available on Pydantic-AI.
26+
27+
To initialize the `OutlinesModel` through the `__init__` method, the first argument you must provide has to be an `outlines.Model` or an `outlines.AsyncModel` instance.
28+
29+
For instance:
30+
31+
```python
32+
import outlines
33+
from transformers import AutoModelForCausalLM, AutoTokenizer
34+
35+
from pydantic_ai.models.outlines import OutlinesModel
36+
37+
outlines_model = outlines.from_transformers(
38+
AutoModelForCausalLM.from_pretrained('erwanf/gpt2-mini'),
39+
AutoTokenizer.from_pretrained('erwanf/gpt2-mini')
40+
)
41+
model = OutlinesModel(outlines_model)
42+
```
43+
44+
Alternatively, you can use some `OutlinesModel` class methods made to load a specific type of Outlines model directly. To do so, you must provide as argument the same arguments you would have given to the associated Outlines model loading function.
45+
46+
For instance:
47+
48+
```python
49+
from transformers import AutoModelForCausalLM, AutoTokenizer
50+
51+
from pydantic_ai.models.outlines import OutlinesModel
52+
53+
model = OutlinesModel.from_transformers(
54+
AutoModelForCausalLM.from_pretrained('erwanf/gpt2-mini'),
55+
AutoTokenizer.from_pretrained('erwanf/gpt2-mini')
56+
)
57+
```
58+
59+
There are methods for the 6 Outlines models that are officially supported in the integration into Pydantic-AI:
60+
- `from_transformers`
61+
- `from_llama_cpp`
62+
- `from_mlxlm`
63+
- `from_sglang`
64+
- `from_dottxt`
65+
- `from_vllm_offline`
66+
67+
As you already providing an Outlines model instance, there is no need to provide an `OutlinesProvider` yourself.
68+
69+
## Running the model
70+
71+
Once you have initialized an `OutlinesModel`, you can use it with an Agent as with all other Pydantic-AI models.
72+
73+
Outlines does not support tools yet, but support for that feature will be added in the near future.

docs/models/overview.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Pydantic AI is model-agnostic and has built-in support for multiple model provid
1010
* [Cohere](cohere.md)
1111
* [Bedrock](bedrock.md)
1212
* [Hugging Face](huggingface.md)
13+
* [Outlines](outlines.md)
1314

1415
## OpenAI-compatible Providers
1516

docs/thinking.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,6 @@ agent = Agent(model, model_settings=settings)
8989
...
9090
```
9191

92-
## Mistral / Cohere
92+
## Mistral / Cohere / Outlines
9393

94-
Neither Mistral nor Cohere generate thinking parts.
94+
Neither Mistral, nor Cohere, nor Outlines generate thinking parts.

0 commit comments

Comments
 (0)