Support 3D Weights in GPTQ Algorithm

[anzr299]

Changes

The approach here is quite straightfoward, _quantize_weights works as usual for 2D weights. The difference is in calculate hessian where the hessian is 3D in both 3D and 2D weights case. By default hessian has the shape (1, hidden_dim, hidden_dim).
Before this was just (hidden_dim, hidden_dim). For 3D, it is (num_experts/batch, hidden_dim, hidden_dim).

Now, this 3D hessian or "batched" hessian is looped over and the 2D weight is extracted and passed to the old _quantize_weights function as usual and scale/zp are returned. These scales and zp are then stacked together in a collector variable. For 2D case, it is flattened. For 3D the stacked scale, zp are returned.

NOTE: Scale Estimation + GPTQ support is not added for 3D weights yet

Reason for changes

Support 3D weights for models like MoE in GPTQ

Related tickets

175789 & 175212

Tests

Model: Qwen/Qwen3-30B-A3B
NNCF Backend: OpenVINO
Higher is better.
Task: gsm8k
Limit: 100
Max New Tokens: 10000
OpenVINO version: 2026.0.0.dev20251111 (with WA for 176465)
n-shots: 5(default)

Precision Type	Filter	Value
INT4 SYM GS128 (with GPTQ) Calibrated on GSM8k with 128 samples	flexible-extract	0.79
	strict-match	0.64
INT4 SYM GS128 (with GPTQ after bug fix commit dc355fe) Calibrated on GSM8k with 128 samples	flexible-extract	0.78
	strict-match	0.57
INT4 SYM GS128	flexible-extract	0.55
	strict-match	0.29
FP32	flexible-extract	0.92
	strict-match	0.82

Comparison of accuracy with meta-llama/Llama-3.2-1B-Instruct on Develop and this branch

Variant	bits_per_byte	byte_perplexity	word_perplexity
This Branch (GPTQ)	0.7965	1.7368	19.1466
develop (GPTQ)	0.7965	1.7368	19.1466

[andreyanufr]

@AlexanderDokuchaev Is it possible to use np here ?

[anzr299]

if not I can use the same approach as

nncf/src/nncf/quantization/algorithms/weight_compression/activation_stats.py

Line 46 in d35c32b

reduce(mul, shape[:act_ch_axis] + shape[act_ch_axis % len(shape) + 1 :], 1) for shape in stats.shape_values

[anzr299]

Done

[andreyanufr]

In which model does this happen?

[anzr299]

For now this is just a safety check. Since for 3D case also we pass 2D hessian to this function. I added it when the older test called this function manually. Would it be better to remove it?

[Copilot]

Pull request overview

This PR adds support for 3D weights in the GPTQ algorithm to enable quantization of models with 3D weight tensors, such as Mixture-of-Experts (MoE) models. The implementation extends the existing GPTQ algorithm to handle batched Hessian matrices.

Changes:

• Extended _calculate_hessian to support 3D weight tensors by creating batched Hessian matrices
• Refactored _quantize_weights to accept tensors directly instead of fetching them, enabling batch processing
• Added loop-based quantization in apply method to process each batch of 3D weights separately

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
src/nncf/quantization/algorithms/weight_compression/gptq.py	Implements core GPTQ algorithm changes to support 3D weights through batched Hessian calculation and iterative quantization
tests/openvino/native/quantization/test_gptq.py	Adds parameterized test coverage for both 2D and 3D weight cases with reference implementation validation

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ljaljushkin]

Nice addition!