Update to transformers v5

[hmellor]

Changes:

• Update Transformers pin to 5.0.0
• Update Tokenizers pin to 0.22.2 (as is required by Transformers 5.0.0)
• Update PEFT lower bound to 0.18.1 so that huggingface/peft@41c07f0 is included (guards import of HybridCache on Transformers version)
• Update Accelerate pin to 1.1.0 so that 4-bit bnb can work on Transformers v5
• Update Mamba pin to 2.3.0 so that state-spaces/mamba@35e927b is included (removes import that was deleted in Transformers v5)
• Replace HF_HUB_ENABLE_HF_TRANSFER with HF_XET_HIGH_PERFORMANCE as the HF Hub is all Xet now so hf_transfer doesn't do anything anymore
• Add HF_HUB_DOWNLOAD_TIMEOUT=60 to the CI environment to deal with the shortened timeout in huggingface-hub==1 since it switched to httpx
• Adds a backward compatbility tests that runs the same tests as "Transformers nightly", but with 4.57.5 installed

Architectures/models that will no longer work after the upgrade:

• MiniCPMV - Custom processing code on the Hub is incompatible with Transformers v5 (PR made but unmerged)
• Molmo2ForConditionalGeneration - Molmo2Processor uses deprecated optional_attributes and passes arbitrary kwargs to ProcessorMixin.__init__ which is no longer supported in Transformers v5
• OpenCUAForConditionalGeneration - Custom code is not compatible with Transformers v5
• OpenPanguVLForConditionalGeneration - OpenPanguVLVideoProcessorInitKwargs does not specify total=False, making all kwargs required
• HCXVisionForCausalLM - Custom model code imports something deprecated that was deleted in Transformers v5

Caution

30d8b3d must be reverted before this can be merged

---

Supplementary PRs:

• Transformers:
  • Fix failing tests due to no attribute pad_token_id huggingface/transformers#43453
  • custom tok init fix huggingface/transformers#43591
  • Fixes configuration default values huggingface/transformers#43592
  • [vllm compat] Separate renaming from conversion ops huggingface/transformers#43621
  • Allow Attention and Experts to be used as standalone modules huggingface/transformers#43622
  • Don't modify tied_weight_keys in-place huggingface/transformers#43619
  • Tie zamba weights correctly huggingface/transformers#43623
  • Fix accelerate integration import  huggingface/transformers#43732
  • Fix convert_rope_params_to_dict so it uses rope_theta from the config huggingface/transformers#43766
  • Fix init weights in remote code huggingface/transformers#43768
  • [MistralCommonBackend] fix loading proc huggingface/transformers#43887
  • [Jamba] Fallback to slow path and warn instead of error out huggingface/transformers#43889
  • adding BC for custom toks accessing slow tok attrs deprecated in v5 huggingface/transformers#43898
• vLLM:
  • Remove head_mask from Ultravox and Swin #30764
  • Fix instantiation of HfHubHTTPError in LoRA test #30768
  • Don't assume position_embedding_type will be present for BERT and RoBERTa models #30770
  • Fix weight mapping test for Transfomers v5 #33162
  • Use Transformers v5 WeightRenaming for Transformers modeling backend #31545
  • Don't use min_pixels/max_pixels from Qwen2VL's processor #33208
  • Fix tie_word_embeddings for multimodal models in Transformers v5 #33359
  • Explicitly set return_dict for apply_chat_template #33372
  • Remove incorrect tokenizer info test #33565
  • Fix Gemma3n audio encoder for Transformers v5 #33673
  • Fix offline test for Transformers v5 #33682
  • Fix Gemma3 GGUF for Transformers v5 #33683
  • Fix tokenizer test for renamed attr on Transformers v5 #33902
  • [Bugfix] Fix models and tests for transformers v5 #33977
  • Bump lm-eval version for Transformers v5 compatibility #33994
  • Bump mamba-ssm version in CI for Transformers v5 compatibility #34233
  • Stop testing for slow tokenizers as they will not exist soon #34235
  • Make Qwen3VL compatible with Transformers v5 #34262
  • Make JAIS compatible with Transformers v5 #34264
  • [Bugfix] Fix more multimodal tests for transformers V5 #34334
  • Don't try and run GLM-ASR with remote code #34352
  • [Bugfix] Standardize getting number of image patches/tokens #34358
  • Fix MoE for the Transformers modelling backend #34436
• Model repos:
  • https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct/discussions/19
  • https://huggingface.co/xlangai/OpenCUA-7B/discussions/9
  • (unmerged) https://huggingface.co/xlangai/OpenCUA-7B/discussions/10
  • https://huggingface.co/moonshotai/Kimi-VL-A3B-Instruct/discussions/34
  • https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2/discussions/14
  • (unmerged) https://huggingface.co/nvidia/llama-nemotron-rerank-1b-v2/discussions/10
  • (unmerged) https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5/discussions/78
  • (unmerged)https://huggingface.co/FreedomIntelligence/openPangu-VL-7B/discussions/2
• Other:
  • Make this fork Transformers v5 compatible robertgshaw2-redhat/lm-evaluation-harness#1
  • Fix call to modify_gen_kwargs in vllm_vlms.py EleutherAI/lm-evaluation-harness#3573

[gemini-code-assist]

Code Review

This pull request aims to update the transformers library to version 5. The changes correctly update the version in requirements/test.in and requirements/nightly_torch_test.txt, and also add the --pre flag to uv pip install in the Dockerfile to allow installation of the release candidate. However, there is a critical oversight: requirements/common.txt still contains a constraint transformers < 5. This will lead to build failures for any configuration that relies on common.txt. This file must be updated to allow transformers v5 for this PR to be mergeable.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

[mergify]

Documentation preview: https://vllm--30566.org.readthedocs.build/en/30566/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

Hi @hmellor, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

Hi @hmellor, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint