[Model] Support Qwen3.5?

[aabbccddwasd]

[dsikka]

Hi @aabbccddwasd - we will be looking into this this coming week!

[aabbccddwasd]

I'm working on it, but I don't have enough vram to test

[Sehyo]

Made a pull request and I am currently uploading a nvfp4 qwen 3.5 quant to Huggingface made with my PR.

[paulplay-pm]

Hi @aabbccddwasd - we will be looking into this this coming week!

does it resolved？pls
how to compress qwen3.5 model?

[youki-sada]

@Sehyo Thank you for your excellent pull request. I was able to successfully create a quantized Qwen3.5 35B w4a4 NVFP4 model using this PR #2383 .
On the other hand, when I ran examples/quantization_w4a4_fp4/qwen3_5_moe_example.py in my environment, I encountered an issue with w4a4 quantization targeting Blackwell for the 122B model: the reasoning output was broken and consisted only of repeated "!" characters.
Notably, the HF model you uploaded with w4a4 quantization, as well as my locally generated w4a16 122B model, do not exhibit this issue.
Could you please let me know what hardware environment you used when running PTQ? Any additional advice or suggestions would be greatly appreciated.

[youki-sada]

This is my environment.

• server1: PRO6000 x2, server2: L40S x8
• Calibration was performed on CPU due to insufficient VRAM. model = AutoModelForImageTextToText.from_pretrained(MODEL_ID, dtype="auto", device_map='cpu')
• vllm==0.17.1+cu130
• compressed-tensors==0.14.1.dev8+g16ce668
• Calibration
  • Samples: 512 (256 from each dataset)
    • Datasets:
      • HuggingFaceH4/ultrachat_200k (train_sft split)
      • nvidia/Nemotron-Post-Training-Dataset-v2 (chat split)
  • Max sequence length: 4096
  • All experts calibrated: moe_calibrate_all_experts=True

[mccatec]

Is there any fp8 quant example available? Is calibration necessary for Qwen3.5?

[dsikka]

HI all,

We have multiple checkpoints released and available:

https://huggingface.co/RedHatAI/Qwen3.5-122B-A10B-NVFP4
https://huggingface.co/RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic
https://huggingface.co/RedHatAI/Qwen3.5-122B-A10B-FP8-dynamic

Support for this model: #2482

If you have any other issues related to a specific model, please open a new issue

[halexan]

HI all,

We have multiple checkpoints released and available:

https://huggingface.co/RedHatAI/Qwen3.5-122B-A10B-NVFP4 https://huggingface.co/RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic https://huggingface.co/RedHatAI/Qwen3.5-122B-A10B-FP8-dynamic

Support for this model: #2482

If you have any other issues related to a specific model, please open a new issue

Hi @dsikka !

Any creation details for fp8 dynamic? I am looking to repro by myself.