Avoid eager transformers/torch import in levanter#2942
Open
Conversation
Importing anything from levanter eagerly imported transformers, which unconditionally imports torch and crashes on CPU-only environments missing CUDA libs. Three independent chains were fixed: 1. hf_utils.py: guard transformers import under TYPE_CHECKING (the HfTokenizer alias is only used in annotations, never at runtime) 2. silence_transformer_nag(): remove redundant `import transformers` after setting the env var 3. levanter/__init__.py: lazy-load eval and eval_harness, which have a hard runtime dependency on transformers via _batch_tokenizer.py Fixes #2941 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
yonromai
force-pushed
the
fix/lazy-transformers-import
branch
from
a330612 to
6445967
Compare
2 tasks
rjpower
approved these changes
Feb 21, 2026
Collaborator
rjpower
left a comment
rjpower
left a comment
There was a problem hiding this comment.
Seems fine but dlwh should be the judge. Dumb question, can we get torch blow up like this? It seems like it should be easy but I couldn't figure it out before so maybe not worth a lot of time.
dlwh
approved these changes
Feb 21, 2026
Member
dlwh
left a comment
dlwh
left a comment
There was a problem hiding this comment.
the real problem is that we're installing cuda wheels on non-cuda machines but this is a fine workaround
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Importing anything from
levantereagerly importstransformers→torch, which crashes on CPU-only environments missing CUDA libs (#2937, #2897).Three independent chains were pulling in
transformers:hf_utils.py—from transformers import ...at module level to defineHfTokenizer. No call site uses this alias at runtime (no isinstance checks), so it's nowTYPE_CHECKING-only withAnyat runtime.silence_transformer_nag()— Didimport transformersafter setting the env var. The env var alone is sufficient.levanter.eval/eval_harness— Reach_batch_tokenizer.pywhich has a hard runtime dependency on transformers. Now lazy-loaded via__getattr__in__init__.py.Fixes #2941.
Test plan
from levanter.layers.rotary import Llama3RotaryEmbeddingsConfigno longer importstransformers🤖 Generated with Claude Code