Transformers v5 compatibility

[Tcc0403]

This is an issue tracker for transformers v5.

To test transformers v5 locally, install v5.0.0rc1 in your local environment.

uv pip install transformers==5.0.0rc1

Known issues

Feel free to report if you find any issue that isn't listed below.

Monkey patch

Errors when running

python3 -m pytest test/transformers/test_monkey_patch.py

See details in #960 (comment)

• AttributeError for 'language_model' in transformers v5 Fix missing property access for multimodal models #966
• KeyError: 'mrope'
• Mixtral: 'MixtralDecoderLayer' object has no attribute 'block_sparse_moe'
• 'Llama4VisionConfig' object has no attribute 'rope_theta'
• 'Gemma3ForConditionalGeneration' object has no attribute 'vision_tower'
• MoE breaking change, see also Moe kernel for latest transformers v5 #958