Skip to content

[Bug]: MXFP LLMC format is not correct. #1629

@xin3he

Description

@xin3he

Problem Description

AutoRound MXFP config is not correct. so I used the config that Yi generated. here.https://huggingface.co/INC4AI/Qwen3-8B-MXFP8-LLMC-TEST-ONLY/blob/main/config.json#L59

For example, the scale dtype in AR is torch.float8_e4m3fn

Reproduction Steps

pytest test_cuda/export/test_llmc_format.py::TestAutoRound::test_mxfp8_llmcompressor_format with branch xinhe/3-27b

Environment Information

No response

Error Logs

"quantization_config": {
    "config_groups": {
      "group_0": {
        "format": null,
        "input_activations": {
          "actorder": null,
          "block_structure": null,
          "dynamic": "local",
          "group_size": 32,
          "is_mx": true,
          "num_bits": 4,
          "observer": "static_minmax",
          "observer_kwargs": {},
          "scale_dtype": "torch.float8_e4m3fn",
          "strategy": "tensor_group",
          "symmetric": true,
          "type": "float",
          "zp_dtype": null
        },
        "output_activations": null,
        "targets": [
          "Linear"
        ],
        "weights": {
          "actorder": null,
          "block_structure": null,
          "dynamic": false,
          "group_size": 32,
          "is_mx": true,
          "num_bits": 4,
          "observer": "memoryless_minmax",
          "observer_kwargs": {},
          "scale_dtype": "torch.float8_e4m3fn",
          "strategy": "tensor_group",
          "symmetric": true,
          "type": "float",
          "zp_dtype": null
        }
      }
    },
    "format": "float-quantized",
    "global_compression_ratio": null,
    "ignore": [],
    "kv_cache_scheme": null,
    "provider": "auto-round",
    "quant_method": "compressed-tensors",
    "quantization_status": "compressed"
  },

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions