|
| 1 | +# Outlines |
| 2 | + |
| 3 | +## Install |
| 4 | + |
| 5 | +To use `OutlinesModel`, you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `outlines` optional group: |
| 6 | + |
| 7 | +```bash |
| 8 | +pip/uv-add "pydantic-ai-slim[outlines]" |
| 9 | +``` |
| 10 | + |
| 11 | +As Outlines is a library allowing you to run model from various different providers, it does not include the necessary dependencies for any model, but instead requires you to install the optional group for the provider you want to use. For instance: |
| 12 | + |
| 13 | +```bash |
| 14 | +pip/uv-add "outlines[transformers]" |
| 15 | +``` |
| 16 | + |
| 17 | +Or |
| 18 | + |
| 19 | +```bash |
| 20 | +pip/uv-add "outlines[mlxlm]" |
| 21 | +``` |
| 22 | + |
| 23 | +## Model Initialization |
| 24 | + |
| 25 | +As Outlines is not an inference provider, but instead a library allowing you to run bith local and API-based models, instantiating the model is a bit different from the other models available on Pydantic-AI. |
| 26 | + |
| 27 | +To initialize the `OutlinesModel` through the `__init__` method, the first argument you must provide has to be an `outlines.Model` or an `outlines.AsyncModel` instance. |
| 28 | + |
| 29 | +For instance: |
| 30 | + |
| 31 | +```python |
| 32 | +import outlines |
| 33 | +from transformers import AutoModelForCausalLM, AutoTokenizer |
| 34 | + |
| 35 | +from pydantic_ai.models.outlines import OutlinesModel |
| 36 | + |
| 37 | +outlines_model = outlines.from_transformers( |
| 38 | + AutoModelForCausalLM.from_pretrained('erwanf/gpt2-mini'), |
| 39 | + AutoTokenizer.from_pretrained('erwanf/gpt2-mini') |
| 40 | +) |
| 41 | +model = OutlinesModel(outlines_model) |
| 42 | +``` |
| 43 | + |
| 44 | +Alternatively, you can use some `OutlinesModel` class methods made to load a specific type of Outlines model directly. To do so, you must provide as argument the same arguments you would have given to the associated Outlines model loading function. |
| 45 | + |
| 46 | +For instance: |
| 47 | + |
| 48 | +```python |
| 49 | +from transformers import AutoModelForCausalLM, AutoTokenizer |
| 50 | + |
| 51 | +from pydantic_ai.models.outlines import OutlinesModel |
| 52 | + |
| 53 | +model = OutlinesModel.from_transformers( |
| 54 | + AutoModelForCausalLM.from_pretrained('erwanf/gpt2-mini'), |
| 55 | + AutoTokenizer.from_pretrained('erwanf/gpt2-mini') |
| 56 | +) |
| 57 | +``` |
| 58 | + |
| 59 | +There are methods for the 6 Outlines models that are officially supported in the integration into Pydantic-AI: |
| 60 | +- `from_transformers` |
| 61 | +- `from_llama_cpp` |
| 62 | +- `from_mlxlm` |
| 63 | +- `from_sglang` |
| 64 | +- `from_dottxt` |
| 65 | +- `from_vllm_offline` |
| 66 | + |
| 67 | +As you already providing an Outlines model instance, there is no need to provide an `OutlinesProvider` yourself. |
| 68 | + |
| 69 | +## Running the model |
| 70 | + |
| 71 | +Once you have initialized an `OutlinesModel`, you can use it with an Agent as with all other Pydantic-AI models. |
| 72 | + |
| 73 | +Outlines does not support tools yet, but support for that feature will be added in the near future. |
0 commit comments