-
Notifications
You must be signed in to change notification settings - Fork 92
Open
Labels
bugSomething isn't workingSomething isn't working
Milestone
Description
Problem Description
AutoRound MXFP config is not correct. so I used the config that Yi generated. here.https://huggingface.co/INC4AI/Qwen3-8B-MXFP8-LLMC-TEST-ONLY/blob/main/config.json#L59
For example, the scale dtype in AR is torch.float8_e4m3fn
Reproduction Steps
pytest test_cuda/export/test_llmc_format.py::TestAutoRound::test_mxfp8_llmcompressor_format with branch xinhe/3-27b
Environment Information
No response
Error Logs
"quantization_config": {
"config_groups": {
"group_0": {
"format": null,
"input_activations": {
"actorder": null,
"block_structure": null,
"dynamic": "local",
"group_size": 32,
"is_mx": true,
"num_bits": 4,
"observer": "static_minmax",
"observer_kwargs": {},
"scale_dtype": "torch.float8_e4m3fn",
"strategy": "tensor_group",
"symmetric": true,
"type": "float",
"zp_dtype": null
},
"output_activations": null,
"targets": [
"Linear"
],
"weights": {
"actorder": null,
"block_structure": null,
"dynamic": false,
"group_size": 32,
"is_mx": true,
"num_bits": 4,
"observer": "memoryless_minmax",
"observer_kwargs": {},
"scale_dtype": "torch.float8_e4m3fn",
"strategy": "tensor_group",
"symmetric": true,
"type": "float",
"zp_dtype": null
}
}
},
"format": "float-quantized",
"global_compression_ratio": null,
"ignore": [],
"kv_cache_scheme": null,
"provider": "auto-round",
"quant_method": "compressed-tensors",
"quantization_status": "compressed"
},Additional Context
No response
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working