[Feature]: FP8 for DS-R1

[nzmora-nvidia]

🚀 The feature, motivation and pitch

• Block-wise GEMM quantization
• MLA + FP8 KV-cache

  TensorRT-LLM/examples/models/core/deepseek_v3/README.md

  Line 748 in c4abca3

  FP8 KV Cache and MLA is supported on Hopper and Blackwell. The accuracy loss is small, with GSM8k accuracy drop less than 1%.

• Note! Different kernel sources for Hopper and Blackwell

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.