Skip to content

outlines[transformers] fails to compile due to setuptools missing #1738

@the-vampiire

Description

@the-vampiire

Describe the issue as clearly as possible:

when using outlines[transformers] with pytorch models, i get torch compilation warnings about missing setuptools. the warnings indicate that pytorch can't compile certain kernels and falls back to slower execution. adding setuptools manually fixes this but it should be included in the transformers extra since pytorch compilation features require it at runtime.

Steps/code to reproduce the bug:

# mkdir /tmp/outlines-transformers; cd /tmp/outlines-transformers; uv init; uv add 'outlines[transformers]'
# paste contents below (from outlines transformers docs) in default hello.py file
# uv run hello.py

import outlines
from transformers import AutoModelForCausalLM, AutoTokenizer
from pydantic import BaseModel
from typing import List

class Character(BaseModel):
    name: str
    age: int
    skills: List[str]

model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct"),
    AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
)

result = model("Create a character.", output_type=Character, max_new_tokens=200, repetition_penalty=0.5)
print(result) # '{"name": "Evelyn", "age": 34, "skills": ["archery", "stealth", "alchemy"]}'
print(Character.model_validate_json(result)) # name=Evelyn, age=34, skills=['archery', 'stealth', 'alchemy']

Expected result:

model should run without any warnings and with full pytorch compilation optimizations enabled

Error message:

W0826 15:14:55.338000 682 .venv/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py:1339] WON'T CONVERT _apply_token_bitmask_inplace_kernel /.../.venv/lib/python3.11/site-packages/outlines_core/kernels/torch.py line 43 
W0826 15:14:55.338000 682 .venv/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py:1339] due to: 
W0826 15:14:55.338000 682 .venv/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py:1339] Traceback (most recent call last):
[... long traceback ...]
W0826 15:14:55.338000 682 .venv/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py:1339] torch._inductor.exc.InductorError: ModuleNotFoundError: No module named 'setuptools'

Outlines/Python version information:

Version information

``` 1.2.3 Python 3.11.11 (main, Jan 14 2025, 23:36:41) [Clang 19.1.6 ] absl-py==2.3.1 accelerate==1.10.1 aiohappyeyeballs==2.6.1 aiohttp==3.12.15 aiosignal==1.4.0 airportsdata==20250811 annotated-types==0.7.0 anyio==4.10.0 async-timeout==5.0.1 attrs==25.3.0 cachetools==5.5.2 certifi==2025.8.3 charset-normalizer==3.4.3 cloudpickle==3.1.1 datasets==4.0.0 dill==0.3.8 diskcache==5.6.3 exceptiongroup==1.3.0 filelock==3.19.1 frozenlist==1.7.0 fsspec==2025.3.0 genson==1.3.0 google-auth==2.40.3 google-genai==1.31.0 h11==0.16.0 hf-xet==1.1.8 httpcore==1.0.9 httpx==0.28.1 huggingface-hub==0.34.4 idna==3.10 iso3166==2.1.1 jinja2==3.1.6 jsonpath-ng==1.7.0 jsonschema==4.25.1 jsonschema-specifications==2025.4.1 langextract==1.0.8 -e file:///Users/vamp/magic/forks/langextract-outlines markupsafe==3.0.2 ml-collections==1.1.0 more-itertools==10.7.0 mpmath==1.3.0 multidict==6.6.4 multiprocess==0.70.16 mypy==1.17.1 mypy-extensions==1.1.0 networkx==3.5 numpy==2.3.2 outlines==1.2.3 outlines-core==0.2.11 packaging==25.0 pandas==2.3.2 pathspec==0.12.1 pillow==11.3.0 ply==3.11 propcache==0.3.2 psutil==7.0.0 pyarrow==21.0.0 pyasn1==0.6.1 pyasn1-modules==0.4.2 pydantic==2.11.7 pydantic-core==2.33.2 python-dateutil==2.9.0.post0 python-dotenv==1.1.1 pytz==2025.2 pyyaml==6.0.2 referencing==0.36.2 regex==2025.7.34 requests==2.32.5 rpds-py==0.27.0 rsa==4.9.1 ruff==0.12.10 safetensors==0.6.2 setuptools==80.9.0 six==1.17.0 sniffio==1.3.1 sympy==1.14.0 tenacity==9.1.2 tokenizers==0.21.4 torch==2.8.0 tqdm==4.67.1 transformers==4.55.4 typing-extensions==4.15.0 typing-inspection==0.4.1 tzdata==2025.2 urllib3==2.5.0 websockets==15.0.1 xxhash==3.5.0 yarl==1.20.1 ```

Context for the issue:

i believe this affects performance since pytorch can't compile the optimized kernels and falls back to slower execution. users installing outlines[transformers] expect all necessary dependencies for optimal performance to be included.

the issue will affect anyone using uv since it only includes what is declared in dependencies. since outlines is specifically using pytorch compilation features, setuptools should be declared as a dependency in the transformers extra.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions