[Question] Why ignore mlp projection weight in `convert_hf_to_int4.py`

[skpig]

Your Question

slime/tools/convert_hf_to_int4.py

Lines 72 to 80 in bd70add

	ignore_patterns = [
	"re:.*lm_head.*",
	"re:.*norm.*",
	"re:.*embed.*",
	"re:.*self_attn.*",
	"re:.*shared_experts.*",
	"re:.*mlp\\.(gate|up|gate_up|down)_proj.*",
	"re:.*mlp\\.gate\\.*",
	]

I'm not familiar to quantization. Could you please explain the reason why these parameters are ignored during INT4 quantization?

What I've Tried

N/A

Environment (if relevant)

• slime version:
• Python version:
• PyTorch version:
• CUDA/ROCm version:
• GPU type and count:
• OS:

Additional Context

No response

Pre-submission Checklist

• I have read the CONTRIBUTING.md and understand the collaboration scope.
• I have read the documentation and FAQ and my question is not answered there.
• I have searched for existing issues and my question has not been asked before.

[Gao016]

1. lm_head: Preserves logit precision, critical for accurate policy gradients and KL divergence in RL.
2. norm: Essential for numerical stability; quantizing these offers negligible memory gains but high risk.
3. embed: Protects the input semantic space and prevents information loss for rare tokens.
4. self_attn: Prevents "attention drift" in complex reasoning (especially in MLA architectures).
5. shared_experts: Protects the core "common knowledge" base of the MoE model.
6. mlp.(gate|up|gate_up|down)_proj: Maintains the base reasoning capability of dense Transformer blocks.
7. mlp.gate: Protects the MoE Router; any precision loss here leads to incorrect token routing.

By applying these ignore rules, only the routed expert weights are quantized. Since these experts represent the vast majority of the model's total parameters, this strategy maximizes memory efficiency while preserving core stability.
Reference: https://huggingface.co/moonshotai/Kimi-K2-Thinking/tree/main

[skpig]

Thanks for your response!