Skip to content

Commit 69af79c

Browse files
Yatimaibrian-dellabettaHDCharles
authored
Fix SmoothQuant regex to match q_a_proj in DeepSeek/GLM-5 (#2421)
Fixes #2420 The balance_layers pattern re:.*q_proj in DEEPSEEK_V2_SMOOTHQUANT_MAPPINGS does not match q_a_proj (used by DeepSeek V2/V3 and GLM-5). Changed to re:.*q(_a)?_proj$ as suggested by @brian-dellabetta. SUMMARY: Fix regex pattern in DEEPSEEK_V2_SMOOTHQUANT_MAPPINGS to cover both q_proj and q_a_proj. TEST PLAN: Verified with Python regex that the new pattern matches both layer names: re.fullmatch(".*q(_a)?_proj$", "model.layers.0.self_attn.q_proj") -> match re.fullmatch(".*q(_a)?_proj$", "model.layers.0.self_attn.q_a_proj") -> match Signed-off-by: gillesturpin <turpingilles@orange.fr> Co-authored-by: Brian Dellabetta <brian-dellabetta@users.noreply.github.com> Co-authored-by: HDCharles <39544797+HDCharles@users.noreply.github.com>
1 parent 9e9ae3d commit 69af79c

File tree

1 file changed

+1
-1
lines changed
  • src/llmcompressor/modifiers/transform/smoothquant

1 file changed

+1
-1
lines changed

src/llmcompressor/modifiers/transform/smoothquant/utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@
6161

6262
DEEPSEEK_V2_SMOOTHQUANT_MAPPINGS: list[LayerMap] = [
6363
LayerMap(
64-
balance_layers=["re:.*q_proj", "re:.*kv_a_proj_with_mqa"],
64+
balance_layers=["re:.*q(_a)?_proj$", "re:.*kv_a_proj_with_mqa"],
6565
smooth_layers="re:.*input_layernorm",
6666
),
6767
]

0 commit comments

Comments
 (0)