|
| 1 | +# Outlines |
| 2 | + |
| 3 | +## Install |
| 4 | + |
| 5 | +To use [`OutlinesModel`][pydantic_ai.models.OutlinesModel], you need to either install `pydantic-ai`, or install `pydantic-ai-slim` with the `outlines` optional group: |
| 6 | +```bash |
| 7 | +pip/uv-add "pydantic-ai-slim[outlines]" |
| 8 | +``` |
| 9 | + |
| 10 | +As Outlines is a library allowing you to run model from various different providers, it does not include the necessary dependencies for any model, but instead requires you to install the optional group for the provider you want to use. To do so, you can install `pydantic-ai-slim` with an optional group composed of outlines, a dash, and the name of the model. For instance: |
| 11 | + |
| 12 | +```bash |
| 13 | +pip/uv-add "pydantic-ai-slim[outlines-transformers]" |
| 14 | +``` |
| 15 | + |
| 16 | +Or |
| 17 | + |
| 18 | +```bash |
| 19 | +pip/uv-add "pydantic-ai-slim[outlines-mlxlm]" |
| 20 | +``` |
| 21 | + |
| 22 | +There are 5 optional groups for the 5 models supported through Outlines: |
| 23 | + |
| 24 | +- `outlines-transformers` |
| 25 | +- `outlines-llamacpp` |
| 26 | +- `outlines-mlxlm` |
| 27 | +- `outlines-sglang` |
| 28 | +- `outlines-vllm-offline` |
| 29 | + |
| 30 | +## Model Initialization |
| 31 | + |
| 32 | +As Outlines is not an inference provider, but instead a library allowing you to run bith local and API-based models, instantiating the model is a bit different from the other models available on Pydantic AI. |
| 33 | + |
| 34 | +To initialize the `OutlinesModel` through the `__init__` method, the first argument you must provide has to be an `outlines.Model` or an `outlines.AsyncModel` instance. |
| 35 | + |
| 36 | +For instance: |
| 37 | + |
| 38 | +```python {test="skip_ci"} |
| 39 | +import outlines |
| 40 | +from transformers import AutoModelForCausalLM, AutoTokenizer |
| 41 | + |
| 42 | +from pydantic_ai.models.outlines import OutlinesModel |
| 43 | + |
| 44 | +outlines_model = outlines.from_transformers( |
| 45 | + AutoModelForCausalLM.from_pretrained('erwanf/gpt2-mini'), |
| 46 | + AutoTokenizer.from_pretrained('erwanf/gpt2-mini') |
| 47 | +) |
| 48 | +model = OutlinesModel(outlines_model) |
| 49 | +``` |
| 50 | + |
| 51 | +Alternatively, you can use some `OutlinesModel` class methods made to load a specific type of Outlines model directly. To do so, you must provide as argument the same arguments you would have given to the associated Outlines model loading function. |
| 52 | + |
| 53 | +For instance: |
| 54 | + |
| 55 | +```python {test="skip_ci"} |
| 56 | +from transformers import AutoModelForCausalLM, AutoTokenizer |
| 57 | + |
| 58 | +from pydantic_ai.models.outlines import OutlinesModel |
| 59 | + |
| 60 | +model = OutlinesModel.from_transformers( |
| 61 | + AutoModelForCausalLM.from_pretrained('erwanf/gpt2-mini'), |
| 62 | + AutoTokenizer.from_pretrained('erwanf/gpt2-mini') |
| 63 | +) |
| 64 | +``` |
| 65 | + |
| 66 | +There are methods for the 5 Outlines models that are officially supported in the integration into Pydantic AI: |
| 67 | +- [`from_transformers`][pydantic_ai.models.OutlinesModel.from_transformers] |
| 68 | +- [`from_llamacpp`][pydantic_ai.models.OutlinesModel.from_llamacpp] |
| 69 | +- [`from_mlxlm`][pydantic_ai.models.OutlinesModel.from_mlxlm] |
| 70 | +- [`from_sglang`][pydantic_ai.models.OutlinesModel.from_sglang] |
| 71 | +- [`from_vllm_offline`][pydantic_ai.models.OutlinesModel.from_vllm_offline] |
| 72 | + |
| 73 | +As you already providing an Outlines model instance, there is no need to provide an `OutlinesProvider` yourself. |
| 74 | + |
| 75 | +## Running the model |
| 76 | + |
| 77 | +Once you have initialized an `OutlinesModel`, you can use it with an Agent as with all other Pydantic AI models. |
| 78 | + |
| 79 | +Outlines does not support tools yet, but support for that feature will be added in the near future. |
| 80 | + |
| 81 | +## Model Settings |
| 82 | + |
| 83 | +As Outlines can be used with various models that do not share the same model settings, the [OutlinesModelSettings][pydantic_ai.models.OutlinesModel.OutlinesModelSettings] class is a bit different from that of the other Pydantic AI models. |
| 84 | + |
| 85 | +There are no Outlines-specific settings added to the class, you are instead in charge of providing the settings needed for the model of your choice. The standard settings inherited from [ModelSettings][pydantic_ai.settings.ModelSettings] are mapped to the corresponding argument for your model when possible, otherwise they are ignored with a warning. |
0 commit comments