[Bug] fp8 weight quantization only model dispatched to fp8_linear after #11000

[urlesistiana]

After #11000

https://github.com/comfyanonymous/ComfyUI/blob/fc657f471a29d07696ca16b566000e8e555d67d1/comfy/ops.py#L601-L604

The check of "input_scale" at L602 was removed. getattr(self, 'input_scale', None) is not None and

On platform that supports fp8 mm,
if a weight does not have key "input_scale", which is "weight quantization only model",
it will be dispatched to fp8_linear with default "input_scale" = 1,
ends up with huge quality regression.

I'm not sure if this is intended or it is a bug.

[comfyanonymous]

yes this is intended, if you want to disable it there is the "full_precision_matrix_mult" option that can be set per layer that when set to True will do the matrix multiplication in regular precision instead of fp8.

[urlesistiana]

Thanks for the info. I noticed the "full_precision_matrix_mult" in code but didn't realize it can be configured in model metadata.

QUANTIZATION.md might need a update.