[FEA] [CuTeDSL] Various Dtype support

[HaoKang-Timmy]

Which component requires the feature?

CuTe DSL

Feature Request

Hi, thanks for your great work.

I wonder if CuTeDSL plans to support FP4 and int8 in the future?

FP4 is widely used in inference on blackwell architecture

And Int8 is widely used for model quantization that decreases the memory footprint of inference and training.

Those are quite important features for algorithm designers.

[vickiw973]

Our released Blackwell gemm support int8 data type
https://github.com/NVIDIA/cutlass/blob/main/examples/python/CuTeDSL/blackwell/dense_gemm_persistent.py#L100
And the latest release also have the blocked scaled gemm which supports nvfp4
https://github.com/NVIDIA/cutlass/blob/main/examples/python/CuTeDSL/blackwell/dense_blockscaled_gemm_persistent.py#L107

Are these good for you to try?

[HaoKang-Timmy]

https://github.com/NVIDIA/cutlass/blob/main/examples/python/CuTeDSL/blackwell/dense_gemm_persistent.py#L100

Thanks! One last question. I wonder if int8 is supported on Hopper as well?

[vickiw973]

I wonder if int8 is supported on Hopper as well?

No, the public example doesn't support Int8, we can add this type in the coming release. cc @YangWang92

[HaoKang-Timmy]

I wonder if int8 is supported on Hopper as well?

No, the public example doesn't support Int8, we can add this type in the coming release. cc @YangWang92

Thank you very much. Looking forward to it.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[brandon-yujie-sun]

I wonder if int8 is supported on Hopper as well?

No, the public example doesn't support Int8, we can add this type in the coming release. cc @YangWang92

Thank you very much. Looking forward to it.

@HaoKang-Timmy This is supported in 4.3 dev, please check out https://github.com/NVIDIA/cutlass/blob/main/examples/python/CuTeDSL/hopper/dense_gemm_persistent.py

Please let us know if you see any other issues.