Skip to content

Commit d497b5a

Browse files
authored
[fix] Fix visual layer ignore pattern for Qwen2.5-VL models (#1766)
Qwen2.5-VL uses "model.visual.*" layer names while Qwen2-VL uses "visual.*". Updated ignore patterns to handle both naming conventions correctly. ## SUMMARY: Updated visual layer ignore patterns to support both Qwen2-VL (`visual.*`) and Qwen2.5-VL (`model.visual.*`) naming conventions, ensuring proper exclusion of visual layers from quantization in both model versions. ### Problem Different Qwen VL model versions use different naming conventions for visual layers: - **Qwen2-VL**: Uses `visual.*` pattern (e.g., `visual.blocks.0.attn.qkv`) - **Qwen2.5-VL**: Uses `model.visual.*` pattern (e.g., `model.visual.blocks.0.attn.qkv`) The current ignore pattern `"re:visual.*"` only works for Qwen2-VL but fails for Qwen2.5-VL models, causing visual layers to be incorrectly included in quantization. ### Solution Updated ignore patterns to handle both naming conventions: - Keep `"re:visual.*"` for Qwen2-VL compatibility - Add `"re:model.visual.*"` for Qwen2.5-VL compatibility ## TEST PLAN: Verified that `"re:model.visual.*"` pattern matches Qwen2.5-VL layer names
1 parent ba72530 commit d497b5a

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/quantization_w8a8_fp8/qwen_2_5_vl_example.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
recipe = QuantizationModifier(
1818
targets="Linear",
1919
scheme="FP8_DYNAMIC",
20-
ignore=["re:.*lm_head", "re:visual.*"],
20+
ignore=["lm_head", "re:visual.*", "re:model.visual.*"],
2121
)
2222

2323
# Apply quantization and save to disk in compressed-tensors format.

0 commit comments

Comments
 (0)