Skip to content

Commit 3e2cf4b

Browse files
Varun Sundar Rabindranathbnellnm
authored andcommitted
zero out attn outputs during profile run
Signed-off-by: Varun Sundar Rabindranath <[email protected]> Signed-off-by: Bill Nell <[email protected]>
1 parent 0851b31 commit 3e2cf4b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/v1/attention/backends/mla/common.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -882,7 +882,7 @@ def forward(
882882

883883
if attn_metadata is None:
884884
# Profiling run.
885-
return output
885+
return output.fill_(0)
886886

887887
num_actual_toks = attn_metadata.num_actual_tokens
888888

0 commit comments

Comments
 (0)