[Question] Why delete q_b_scale kv_b_scale k_b_trans_scale

[bobbych94]

Why did the gpt_attention function delete the parameters

q_b_scale: Optional[Tensor] = None,
kv_b_scale: Optional[Tensor] = None,
k_b_trans_scale: Optional[Tensor] = None,

and is_fp8_model_flag in the latest code?
Can anyone explain the reason?

[karljang]

Apologies for the very delayed response.

• The old FP8 scaling parameters (q_b_scale, kv_b_scale, k_b_trans_scale, is_fp8_model_flag) were part of the TensorRT plugin implementation for MLA.
• And the TensorRT backend (using trtllm-build and the GPTAttentionPlugin) no longer supports MLA used by DeepSeek models (V2, V3, R1). MLA support has been moved exclusively to the PyTorch backend.

I hope this comment answers your question.
Closing this issue.