What is the meaning of weight_quantizer in W4A8_AWQ_BETA_CFG?

[ZhuJiaqi9905]

The config of W4A8_AWQ is:

W4A8_AWQ_BETA_CFG = {
    "quant_cfg": {
        "*weight_quantizer": [
            {"num_bits": 4, "block_sizes": {-1: 128, "type": "static"}, "enable": True},
            {"num_bits": (4, 3), "axis": None, "enable": True},
        ],
        "*input_quantizer": {"num_bits": (4, 3), "axis": None, "enable": True},
        **_default_disabled_quantizer_cfg,
    },
    "algorithm": "awq_lite",
}

Why the weight quantizer has two config? Does it mean that the bf16 weight will be first per-tensor quantized to FP8, than the FP8 weight will be per-group quantized to int4 weight?

[i-riyad]

The weight_quantizer list contains two quantization configurations because the quantizer being used is a SequentialQuantizer. This enables multi-stage quantization, where the same tensor can be quantized in different ways for storage and compute.

In this specific config:

"weight_quantizer": [
    {"num_bits": 4, "block_sizes": {-1: 128, "type": "static"}, "enable": True},
    {"num_bits": (4, 3), "axis": None, "enable": True},
]

Here’s what happens:
BF16 weights → Quantized to INT4 for storage (e.g., blockwise Q) → Dequantized and requantized to FP8 (E4M3) for compute → Passed to GEMM (e.g., wgmma or equivalent FP8-friendly kernel)