chore(deps): update dependency transformers to v4.52.1 [security]

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
transformers	==4.28.1 -> ==4.52.1

---

Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.

GitHub Vulnerability Alerts

CVE-2023-2800

Insecure Temporary File in GitHub repository huggingface/transformers 4.29.2 and prior. A fix is available at commit 80ca92470938bbcc348e2d9cf4734c7c25cb1c43 and has been released as part of version 4.30.0.

CVE-2023-7018

Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.

CVE-2023-6730

Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.0.

CVE-2024-3568

The huggingface/transformers library is vulnerable to arbitrary code execution through deserialization of untrusted data within the load_repo_checkpoint() function of the TFPreTrainedModel() class. Attackers can execute arbitrary code and commands by crafting a malicious serialized payload, exploiting the use of pickle.load() on data from potentially untrusted sources. This vulnerability allows for remote code execution (RCE) by deceiving victims into loading a seemingly harmless checkpoint during a normal training process, thereby enabling attackers to execute arbitrary code on the targeted machine.

CVE-2024-11392

Hugging Face Transformers MobileViTV2 Deserialization of Untrusted Data Remote Code Execution Vulnerability. This vulnerability allows remote attackers to execute arbitrary code on affected installations of Hugging Face Transformers. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.

The specific flaw exists within the handling of configuration files. The issue results from the lack of proper validation of user-supplied data, which can result in deserialization of untrusted data. An attacker can leverage this vulnerability to execute code in the context of the current user. Was ZDI-CAN-24322.

CVE-2024-11393

Hugging Face Transformers MaskFormer Model Deserialization of Untrusted Data Remote Code Execution Vulnerability. This vulnerability allows remote attackers to execute arbitrary code on affected installations of Hugging Face Transformers. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.

The specific flaw exists within the parsing of model files. The issue results from the lack of proper validation of user-supplied data, which can result in deserialization of untrusted data. An attacker can leverage this vulnerability to execute code in the context of the current user. Was ZDI-CAN-25191.

CVE-2024-11394

Hugging Face Transformers Trax Model Deserialization of Untrusted Data Remote Code Execution Vulnerability. This vulnerability allows remote attackers to execute arbitrary code on affected installations of Hugging Face Transformers. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.

The specific flaw exists within the handling of model files. The issue results from the lack of proper validation of user-supplied data, which can result in deserialization of untrusted data. An attacker can leverage this vulnerability to execute code in the context of the current user. Was ZDI-CAN-25012.

CVE-2024-12720

A Regular Expression Denial of Service (ReDoS) vulnerability was identified in the huggingface/transformers library, specifically in the file tokenization_nougat_fast.py. The vulnerability occurs in the post_process_single() function, where a regular expression processes specially crafted input. The issue stems from the regex exhibiting exponential time complexity under certain conditions, leading to excessive backtracking. This can result in significantly high CPU usage and potential application downtime, effectively creating a Denial of Service (DoS) scenario. The affected version is v4.46.3.

CVE-2025-1194

A Regular Expression Denial of Service (ReDoS) vulnerability was identified in the huggingface/transformers library, specifically in the file tokenization_gpt_neox_japanese.py of the GPT-NeoX-Japanese model. The vulnerability occurs in the SubWordJapaneseTokenizer class, where regular expressions process specially crafted inputs. The issue stems from a regex exhibiting exponential complexity under certain conditions, leading to excessive backtracking. This can result in high CPU usage and potential application downtime, effectively creating a Denial of Service (DoS) scenario. The affected version is v4.48.1 (latest).

CVE-2025-2099

A vulnerability in the preprocess_string() function of the transformers.testing_utils module in huggingface/transformers version v4.48.3 allows for a Regular Expression Denial of Service (ReDoS) attack. The regular expression used to process code blocks in docstrings contains nested quantifiers, leading to exponential backtracking when processing input with a large number of newline characters. An attacker can exploit this by providing a specially crafted payload, causing high CPU usage and potential application downtime, effectively resulting in a Denial of Service (DoS) scenario.

CVE-2025-3263

A Regular Expression Denial of Service (ReDoS) vulnerability was discovered in the Hugging Face Transformers library, specifically in the get_configuration_file() function within the transformers.configuration_utils module. The affected version is 4.49.0, and the issue is resolved in version 4.51.0. The vulnerability arises from the use of a regular expression pattern config\.(.*)\.json that can be exploited to cause excessive CPU consumption through crafted input strings, leading to catastrophic backtracking. This can result in model serving disruption, resource exhaustion, and increased latency in applications using the library.

CVE-2025-3264

A Regular Expression Denial of Service (ReDoS) vulnerability was discovered in the Hugging Face Transformers library, specifically in the get_imports() function within dynamic_module_utils.py. This vulnerability affects versions 4.49.0 and is fixed in version 4.51.0. The issue arises from a regular expression pattern \s*try\s*:.*?except.*?: used to filter out try/except blocks from Python code, which can be exploited to cause excessive CPU consumption through crafted input strings due to catastrophic backtracking. This vulnerability can lead to remote code loading disruption, resource exhaustion in model serving, supply chain attack vectors, and development pipeline disruption.

CVE-2025-3777

Hugging Face Transformers versions up to 4.49.0 are affected by an improper input validation vulnerability in the image_utils.py file. The vulnerability arises from insecure URL validation using the startswith() method, which can be bypassed through URL username injection. This allows attackers to craft URLs that appear to be from YouTube but resolve to malicious domains, potentially leading to phishing attacks, malware distribution, or data exfiltration. The issue is fixed in version 4.52.1.

CVE-2025-3933

A Regular Expression Denial of Service (ReDoS) vulnerability was discovered in the Hugging Face Transformers library, specifically within the DonutProcessor class's token2json() method. This vulnerability affects versions 4.51.3 and earlier, and is fixed in version 4.52.1. The issue arises from the regex pattern <s_(.*?)> which can be exploited to cause excessive CPU consumption through crafted input strings due to catastrophic backtracking. This vulnerability can lead to service disruption, resource exhaustion, and potential API service vulnerabilities, impacting document processing tasks using the Donut model.

---

Release Notes

huggingface/transformers (transformers)

v4.52.1: : Qwen2.5-Omni, SAM-HQ, GraniteMoeHybrid, D-FINE, CSM, BitNet, LlamaGuard, TimesFM, MLCD, Janus, InternVL

Compare Source

New models

Qwen2.5-Omni

The Qwen2.5-Omni model is a unified multiple modalities model proposed in Qwen2.5-Omni Technical Report from Qwen team, Alibaba Group.

The abstract from the technical report is the following:

We present Qwen2.5-Omni, an end-to-end multimodal model designed to perceive diverse modalities, including text, images, audio, and video, while simultaneously generating text and natural speech responses in a streaming manner. To enable the streaming of multimodal information inputs, both audio and visual encoders utilize a block-wise processing approach. This strategy effectively decouples the handling of long sequences of multimodal data, assigning the perceptual responsibilities to the multimodal encoder and entrusting the modeling of extended sequences to a large language model.

Such a division of labor enhances the fusion of different modalities via the shared attention mechanism. To synchronize the timestamps of video inputs with audio, we organized the audio and video sequentially in an interleaved manner and propose a novel position embedding approach, named TMRoPE (Time-aligned Multimodal RoPE). To concurrently generate text and speech while avoiding interference between the two modalities, we propose Thinker-Talker architecture.

In this framework, Thinker functions as a large language model tasked with text generation, while Talker is a dual-track autoregressive model that directly utilizes the hidden representations from the Thinker to produce audio tokens as output. Both the Thinker and Talker models are designed to be trained and inferred in an end-to-end manner. For decoding audio tokens in a streaming manner, we introduce a sliding-window DiT that restricts the receptive field, aiming to reduce the initial package delay. Qwen2.5-Omni outperforms the similarly sized Qwen2-VL and Qwen2-Audio in both image and audio capabilities. Furthermore, Qwen2.5-Omni achieves state-of-the-art performance on multimodal benchmarks like Omni-Bench.

Notably, Qwen2.5-Omni is the first open-source model to achieve a level of performance in end-to-end speech instruction following that is comparable to its capabilities with text inputs, as evidenced by benchmarks such as MMLU and GSM8K. As for speech generation, Qwen2.5-Omni’s streaming Talker outperform most existing streaming and non-streaming alternatives in robustness and naturalness.

SAM-HQ

SAM-HQ (High-Quality Segment Anything Model) was proposed in Segment Anything in High Quality by Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu.

The model is an enhancement to the original SAM model that produces significantly higher quality segmentation masks while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability.

SAM-HQ introduces several key improvements over the original SAM model:

1. High-Quality Output Token: A learnable token injected into SAM's mask decoder for higher quality mask prediction
2. Global-local Feature Fusion: Combines features from different stages of the model for improved mask details
3. Training Data: Uses a carefully curated dataset of 44K high-quality masks instead of SA-1B
4. Efficiency: Adds only 0.5% additional parameters while significantly improving mask quality
5. Zero-shot Capability: Maintains SAM's strong zero-shot performance while improving accuracy

The abstract from the paper is the following:

The recent Segment Anything Model (SAM) represents a big leap in scaling up segmentation models, allowing for powerful zero-shot capabilities and flexible prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction quality falls short in many cases, particularly when dealing with objects that have intricate structures. We propose HQ-SAM, equipping SAM with the ability to accurately segment any object, while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability. Our careful design reuses and preserves the pre-trained model weights of SAM, while only introducing minimal additional parameters and computation. We design a learnable High-Quality Output Token, which is injected into SAM's mask decoder and is responsible for predicting the high-quality mask. Instead of only applying it on mask-decoder features, we first fuse them with early and final ViT features for improved mask details. To train our introduced learnable parameters, we compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is only trained on the introduced dataset of 44k masks, which takes only 4 hours on 8 GPUs.

Tips:

• SAM-HQ produces higher quality masks than the original SAM model, particularly for objects with intricate structures and fine details
• The model predicts binary masks with more accurate boundaries and better handling of thin structures
• Like SAM, the model performs better with input 2D points and/or input bounding boxes
• You can prompt multiple points for the same image and predict a single high-quality mask
• The model maintains SAM's zero-shot generalization capabilities
• SAM-HQ only adds ~0.5% additional parameters compared to SAM
• Fine-tuning the model is not supported yet

GraniteMoeHybrid

The GraniteMoeHybrid model builds on top of GraniteMoeSharedModel and Bamba. Its decoding layers consist of state space layers or MoE attention layers with shared experts. By default, the attention layers do not use positional encoding.

D-FINE

The D-FINE model was proposed in D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement by
Yansong Peng, Hebei Li, Peixi Wu, Yueyi Zhang, Xiaoyan Sun, Feng Wu

The abstract from the paper is the following:

We introduce D-FINE, a powerful real-time object detector that achieves outstanding localization precision by redefining the bounding box regression task in DETR models. D-FINE comprises two key components: Fine-grained Distribution Refinement (FDR) and Global Optimal Localization Self-Distillation (GO-LSD).
FDR transforms the regression process from predicting fixed coordinates to iteratively refining probability distributions, providing a fine-grained intermediate representation that significantly enhances localization accuracy. GO-LSD is a bidirectional optimization strategy that transfers localization knowledge from refined distributions to shallower layers through self-distillation, while also simplifying the residual prediction tasks for deeper layers. Additionally, D-FINE incorporates lightweight optimizations in computationally intensive modules and operations, achieving a better balance between speed and accuracy. Specifically, D-FINE-L / X achieves 54.0% / 55.8% AP on the COCO dataset at 124 / 78 FPS on an NVIDIA T4 GPU. When pretrained on Objects365, D-FINE-L / X attains 57.1% / 59.3% AP, surpassing all existing real-time detectors. Furthermore, our method significantly enhances the performance of a wide range of DETR models by up to 5.3% AP with negligible extra parameters and training costs. Our code and pretrained models: this https URL.

CSM

The Conversational Speech Model (CSM) is the first open-source contextual text-to-speech model released by Sesame. It is designed to generate natural-sounding speech with or without conversational context. This context typically consists of multi-turn dialogue between speakers, represented as sequences of text and corresponding spoken audio.

Model Architecture:
CSM is composed of two LLaMA-style auto-regressive transformer decoders: a backbone decoder that predicts the first codebook token and a depth decoder that generates the remaining tokens. It uses the pretrained codec model Mimi, introduced by Kyutai, to encode speech into discrete codebook tokens and decode them back into audio.

The original csm-1b checkpoint is available under the Sesame organization on Hugging Face.

BitNet

Trained on a corpus of 4 trillion tokens, this model demonstrates that native 1-bit LLMs can achieve performance comparable to leading open-weight, full-precision models of similar size, while offering substantial advantages in computational efficiency (memory, energy, latency).

LlamaGuard

Llama Guard 4 is a new multimodal model designed to detect inappropriate content in images and text, whether used as input or generated as output by the model. It’s a dense 12B model pruned from Llama 4 Scout model, and it can run on a single GPU (24 GBs of VRAM). It can evaluate both text-only and image+text inputs, making it suitable for filtering both inputs and outputs of large language models. This enables flexible moderation pipelines where prompts are analyzed before reaching the model, and generated responses are reviewed afterwards for safety. It can also understand multiple languages.

TimesFM

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model proposed in A decoder-only foundation model for time-series forecasting by Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. It is a decoder only model that uses non-overlapping patches of time-series data as input and outputs some output patch length prediction in an autoregressive fashion.

The abstract from the paper is the following:

Motivated by recent advances in large language models for Natural Language Processing (NLP), we design a time-series foundation model for forecasting whose out-of-the-box zero-shot performance on a variety of public datasets comes close to the accuracy of state-of-the-art supervised forecasting models for each individual dataset. Our model is based on pretraining a patched-decoder style attention model on a large time-series corpus, and can work well across different forecasting history lengths, prediction lengths and temporal granularities.

MLCD

The MLCD models were released by the DeepGlint-AI team in unicom, which focuses on building foundational visual models for large multimodal language models using large-scale datasets such as LAION400M and COYO700M, and employs sample-to-cluster contrastive learning to optimize performance. MLCD models are primarily used for multimodal visual large language models, such as LLaVA.

Janus

The Janus Model was originally proposed in Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation by DeepSeek AI team and later refined in Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling. Janus is a vision-language model that can generate both image and text output, it can also take both images and text as input.

[!NOTE]
The model doesn't generate both images and text in an interleaved format. The user has to pass a parameter indicating whether to generate text or image.

The abstract from the original paper is the following:

In this paper, we introduce Janus, an autoregressive framework that unifies multimodal understanding and generation. Prior research often relies on a single visual encoder for both tasks, such as Chameleon. However, due to the differing levels of information granularity required by multimodal understanding and generation, this approach can lead to suboptimal performance, particularly in multimodal understanding. To address this issue, we decouple visual encoding into separate pathways, while still leveraging a single, unified transformer architecture for processing. The decoupling not only alleviates the conflict between the visual encoder's roles in understanding and generation, but also enhances the framework's flexibility. For instance, both the multimodal understanding and generation components can independently select their most suitable encoding methods. Experiments show that Janus surpasses previous unified model and matches or exceeds the performance of task-specific models. The simplicity, high flexibility, and effectiveness of Janus make it a strong candidate for next-generation unified multimodal models.

The abstract from the aforementioned Janus-Pro paper, released afterwards, is the following:

In this work, we introduce Janus-Pro, an advanced version of the previous work Janus. Specifically, Janus-Pro incorporates (1) an optimized training strate (2) expanded training data, and (3) scaling to larger model size. With these improvements, Janus-Pro achieves significant advancements in both multimodal understanding and text-to-image instruction-following capabilities, while also enhancing the stability of text-to-image generation. We hope this work will inspire further exploration in the field. Code and models are publicly available.

InternVL

The InternVL3 family of Visual Language Models was introduced in InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models.

The abstract from the paper is the following:

We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm. Rather than adapting a text-only large language model (LLM) into a multimodal large language model (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and linguistic capabilities from both diverse multimodal data and pure-text corpora during a single pre-training stage. This unified training paradigm effectively addresses the complexities and alignment challenges commonly encountered in conventional post-hoc training pipelines for MLLMs. To further improve performance and scalability, InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure. Extensive empirical evaluations demonstrate that InternVL3 delivers superior performance across a wide range of multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its capabilities remain highly competitive with leading proprietary models, including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also maintaining strong pure-language proficiency. In pursuit of open-science principles, we will publicly release both the training data and model weights to foster further research and development in next-generation MLLMs.

Overview of InternVL3 models architecture, which is the same as InternVL2.5. Taken from the original checkpoint.

Comparison of InternVL3 performance on OpenCompass against other SOTA VLLMs. Taken from the original checkpoint.

Kernel integration

We integrate some kernels in the transformers library via the kernels package: https://github.com/huggingface/kernels
We start with some kernels in the Llama model, and we iterate to identify the best performance optimizations

• Llama Kernel integration by @​MekkCyber in #​37092
• [kernels] use original forward at compile time by @​gante in #​37604

TP support

In the previous release, we've added TP support in order to run distributed inference. However, this is not supported for all quantization methods. We are progressively adding support to it. Right now, only compressed-tensors, fp8 and fp8-fbgemm support it.

• Attention Quantization with FBGemm & TP by @​MekkCyber in #​37384
• Restrict & Explain tp_plan for FBgemm by @​MekkCyber in #​37404

Quantization

AutoRound

From the AutoRound contributors:

AutoRound is an advanced quantization algorithm that delivers strong accuracy, even at 2-bit precision. It leverages sign gradient descent to fine-tune both rounding values and min-max clipping thresholds in just 200 steps ... More details here: https://github.com/intel/auto-round

• Add AutoRound quantization support by @​wenhuach21 in #​37393

Quantization Documentation

We have added two new sections to better understand and get started with quantization:

• Quantization concept
• Selecting a quantization method

• Add "selecting a quantization method" doc by @​DerekLiu35 in #​37159
• Update quantization docs by @​DerekLiu35 in #​37439

GGUF

We've added GGUF support to gemma3 family models.

• Add GGUF support to Gemma3 Text backbone by @​Isotr0py in #​37424
• Support loading Gemma3 QAT GGUF models by @​Isotr0py in #​37649

Fast image processors

Most Vision Models and VLMs in Transformers can now benefit from fast image processors. By utilizing torch/torchvision functional transforms, these processors offer a substantial speedup when processing images compared to PiL/numpy functions, and support processing on both CPU and CUDA.

• See the list of updated models: https://github.com/huggingface/transformers/issues/36978
• Learn more about fast image processors: Fast Image Processors

• Add Fast Image Processor for Perceiver by @​rootonchair in #​37176
• Add Fast Image Processor for Flava by @​rootonchair in #​37135
• Add Fast Image Processor for LayoutLMv2 by @​rootonchair in #​37203
• Add Fast Image Processor for LayoutLMv3 by @​rootonchair in #​37201
• Add Fast Image Processor for Donut by @​rootonchair in #​37081
• Add Fast LeViT Processor by @​keetrap in #​37154
• Add Fast Mobilenet-V2 Processor by @​keetrap in #​37113
• Add Fast owlvit Processor by @​keetrap in #​37164
• Add ImageProcessorFast to BiT processor by @​Yann-CV in #​37180
• Add Fast Yolos Processor by @​keetrap in #​37292
• Add Fast Chinese-CLIP Processor by @​keetrap in #​37012
• Add Fast Conditional-DETR Processor by @​keetrap in #​37071
• Fix broken add-fast-image-processor CLI by @​yonigozlan in #​37499
• Bridgetower fast image processor by @​rootonchair in #​37373
• Add Fast Grounding-Dino Processor by @​keetrap in #​37108
• Add Fast PVT Processor by @​keetrap in #​37204
• Add Fast Image Processor for PoolFormer by @​rootonchair in #​37182
• Add Fast Image Processor for MobileNetV1 by @​dmdaksh in #​37111
• Fast image processor for VitMatte added and bug in slow version fixed by @​henrikm11 in #​37616
• [Fast Processor] BEiT by @​ariG23498 in #​37005
• Add Swin2SR ImageProcessorFast by @​thisisiron in #​37169
• Add Fast Image Processor for vilt by @​devxaitist in #​37304

AutoDocstring

The new @auto_docstring decorator makes it easier to add proper documentation when contributing a model without bloating the modeling code:

• [AutoDocstring] Based on inspect parsing of the signature by @​ArthurZucker and @​yonigozlan in https://github.com/huggingface/transformers/pull/33771
• More info on how to use @auto_docstring: AutoDocstring

Custom generate

We now support custom generate methods to be loaded from model.generate. The custom generate methods can be stored on the Hub, enabling quick distribution of experiments regarding new caches, decoding methods, heuristics, ...

from transformers import AutoModelForCausalLM, AutoTokenizer

### `generate` with `custom_generate` -> `generate` uses custom code
### note: calling the custom method prints "✨ using a custom generation method ✨"
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", device_map="auto")

inputs = tokenizer(["The quick brown"], return_tensors="pt").to(model.device)
gen_out = model.generate(**inputs, custom_generate="transformers-community/custom_generate_example", trust_remote_code=True)
print(tokenizer.batch_decode(gen_out, skip_special_tokens=True))

You can find the docs here, and all custom generation methods by searching for the custom_generate tag.

• [generate] Run custom generation code from the Hub by @​gante in #​36405

Chat CLI

The transformers-cli command is updated to be simpler and cleaner, specifically for its chat variant.

The following is now possible and recommended:

transformers chat Qwen/Qwen2.5-3B-Instruct

Additionally, almost any generate flag can now be passed as a positional argument, present and future, as opposed to being limited to a set of hardcoded flags, for example:

transformers chat Qwen/Qwen2.5-0.5B-Instruct do_sample=False max_new_tokens=10

• Transformers cli clean command by @​LysandreJik in #​37657
• [chat] clean code and add base help by @​gante in #​37892
• [chat] generate parameterization powered by GenerationConfig and UX-related changes by @​gante in #​38047

Breaking changes

• 🚨 rm already deprecated pad_to_max_length arg by @​itazap in #​37617
• 🚨🚨🚨 Fix forward of Dinov2ForImageClassification for models with registers by @​psandovalsegura in #​37836
• 🔴 [VLM] Add base model without head by @​zucchini-nlp in #​37033
• 🔴 Video processors as a separate class by @​zucchini-nlp in #​35206
• 🚨🚨 Allow saving and loading multiple "raw" chat template files by @​Rocketknight1 in #​36588
• 🔴 Update CLIP vision attention to new attention interface by @​molbap in #​37498
• 🚨🚨 Setup -> setupclass conversion by @​Rocketknight1 in #​37282

Deprecations

The agents folder is finally removed from transformers in favour of using smolagents.

• [agents] remove agents 🧹 by @​gante in #​37368

We are moving away from torch 2.0 as it has been released more than two years ago.

• byebye torch 2.0 by @​ydshieh in #​37277

General bugfixes and improvements

• fix flex attn when optional args aren't passed by @​winglian in #​37327
• fix llama4 training by @​hiyouga in #​37319
• Fix deepspeed with quantization by @​Cyrilvallez in #​37324
• Fix init empty weights without accelerate by @​Cyrilvallez in #​37337
• Use Python 3.9 syntax in examples by @​cyyever in #​37279
• Fix torchao usage by @​jiqing-feng in #​37034
• enable 2 llama UT cases on xpu by @​yao-matrix in #​37126
• Avoid build crashes when torch.version.xpu doesn't exist and fix Llama4 processor tests by @​Rocketknight1 in #​37346
• fix derived berts _init_weights by @​Cyrilvallez in #​37341
• Update translation template by @​stevhliu in #​37294
• Remove HQQ from caching allocator warmup by @​Cyrilvallez in #​37347
• updated model card for Mistral by @​NahieliV in #​37156
• Update model-card for DINOv2 by @​shubham0204 in #​37104
• Update falcon mamba card by @​ricalanis in #​37253
• Update Model card for GPT2 by @​ash-01xor in #​37101
• Improvements in Gemma2 model card by @​devesh-2002 in #​37076
• Update Model Card for Jamba by @​ParagEkbote in #​37152
• Add bnb to the list of supported quantization methods for LLama4 by @​MekkCyber in #​37348
• Updated Model-card for donut by @​Logeswaran7 in #​37290
• Remove unnecessary attr assignment by @​tugsbayasgalan in #​36837
• more fixes for post-training llama4 by @​winglian in #​37329
• Fixing flex attention for torch=2.6.0 by @​SalmanMohammadi in #​37285
• Multiple llama4 fixe by @​ArthurZucker in #​37353
• Expose blip2qformer by @​alex-jw-brooks in #​37254
• convert float for yarn related arguments in rope_scaling by @​bzantium in #​37139
• Use Python 3.9 syntax in tests by @​cyyever in #​37343
• A bit of cleaning 🧹🧹 by @​Cyrilvallez in #​37215
• fix deepspeed job by @​ydshieh in #​37284
• Set vision config to None for Gemma 1B conversion by @​RyanMullins in #​37366
• [llama 4] dynamic rope decorator by @​gante in #​37365
• Skip non-selected experts for mixtral and qwen2_moe by @​Coco58323 in #​32429
• [core] remove GenerationMixin inheritance by default in PreTrainedModel by @​gante in #​37173
• prune LM Head for USD by @​jmamou in #​36695
• fix(qwen): fix shape error when using tp by @​KimmiShi in #​36947
• Preserve requires_grad in pre quantized model by @​jerryzh168 in #​37354
• Update composition flag usage by @​zucchini-nlp in #​36263
• fix: llama4 conversion script no_rope_layers by @​jmkuebler in #​37359
• update deepspeed docker by @​SunMarc in #​37371
• Fix warning message for PEFT models in text-generation pipeline #​36783 by @​falconlee236 in #​36887
• Apply torchfix to replace deprecated functions: _pytree._register_pytree_node and torch.cpu.amp.autocast by @​bzhong-solink in #​37372
• Fix some failing AWQ tests by @​DerekLiu35 in #​37383
• the fix that did not get in by @​ArthurZucker in #​37370
• handle torch version edge cases by @​winglian in #​37399
• Add warning when failed to acquire other user's lock at model download by @​manueldeprada in #​37395
• Handle torch ver in flexattn by @​Kh4L in #​37400
• Fix Llama4 offset by @​Cyrilvallez in #​37414
• Offloaded hybrid cache for Llama4 by @​Cyrilvallez in #​37401
• mark llama4 as not supported with fa2 by @​winglian in #​37416
• update kernels to 0.4.3 by @​ArthurZucker in #​37419
• Send trainer/fsdp/deepspeed CI job reports to a single channel by @​ydshieh in #​37411
• from_pretrained should handle xpu case by @​sywangyi in #​37382
• Allow rocm systems to run these tests by @​ivarflakstad in #​37278
• use rms_norm_eps for the L2Norm for Llama4 by @​ArthurZucker in #​37418
• [chat-template] Unify tests and clean up 🧼 by @​zucchini-nlp in #​37275
• Fix new failure reports not including anything other than tests/models/ by @​ydshieh in #​37415
• Quark Quantization gated repo by @​MekkCyber in #​37412
• Add image classifier donut & update loss calculation for all swins by @​eljandoubi in #​37224
• Correctly drop tokens in SwitchTransformer by @​mario-aws in #​37123
• Fix require_read_token by @​MekkCyber in #​37422
• fix: use mtime by default in Trainer._rotate_checkpoints with automatic fallback by @​Jerry-Terrasse in #​37260
• (Part 2) feat: allow for tp_size attr for tplizing the model by @​kmehant in #​37054
• Adding to self_comment_ci.yml by @​MekkCyber in #​37426
• [Feat] Support npu in modeling models by @​duanjunwen in #​37369
• Remove old code for PyTorch, Accelerator and tokenizers by @​cyyever in #​37234
• enhance require_deterministic_for_xpu by @​yao-matrix in #​37437
• Fixes: Corrects file path for CUDA kernels by @​DonggeunYu in #​37438
• Simplify soft dependencies and update the dummy-creation process by @​LysandreJik in #​36827
• Update-kernel-pin by @​ArthurZucker in #​37448
• Add moe kernels by @​ArthurZucker in #​37376
• Fix the test fetcher by @​LysandreJik in #​37452
• Remove triton mlp kernel, not compiling for some models by @​MekkCyber in #​37449
• [processor] clean up mulitmodal tests by @​zucchini-nlp in #​37362
• [Regression] Fix Quark quantized model loading after refactorization by @​BowenBao in #​37407
• prevent creating a view/leaf param for low rank optimizers w FSDP by @​winglian in #​37379
• Disable kernels for quantization by @​MekkCyber in #​37446
• Add weights_only=True to torch.load by @​cyyever in #​37062
• Add XPU case to is_torch_bf16_gpu_available by @​cyyever in #​37132
• nit: typing use Llama4TextConfig instead of Llama4Config by @​kmehant in #​37430
• Delete hubconf.py by @​Rocketknight1 in #​37455
• Fix typing issues with SigLip2 by @​EricWiener in #​37356
• fix: (llama4) fix no_split_modules to be picked up for fsdpv1 and v2 sharding by @​kmehant in #​37462
• make test_snowman_image_captioning pass on XPU, by sharing same atol w/ ROCM by @​yao-matrix in #​37480
• Remove fsspec dependency which isn't directly used by transformers by @​cyyever in #​37318
• Fix tests failed with gated repos. by @​ydshieh in #​37484
• [ci] fix doc builder by @​zucchini-nlp in #​37489
• Fixed broken links by @​cypherpepe in #​37466
• Detect and fix most _init_weights() issues - make it work for composite models by @​Cyrilvallez in #​37070
• [bug] deprecated deta load_cuda_kernel, MultiScaleDeformableAttention by @​chagmgang in #​37443
• Fix mask handling for flex attention in llama/gemma2/mistral/qwen2 by @​flukeskywalker in #​37381
• Fix wrong argparse type in modular checker script by @​seven-mile in #​37472
• Fixing gated repo issues by @​MekkCyber in #​37463
• [qwen-omni] fix processor by @​zucchini-nlp in #​37493
• Remove deprecation warning for num_logits_to_keep by [@​Cyrilvallez](https:

---

Configuration

📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[github-actions]

Stale pull request message

[github-actions]

Stale pull request message