Skip to content

Commit 5318218

Browse files
Varun Sundar Rabindranathbnellnm
authored andcommitted
zero out attn outputs during profile run
Signed-off-by: Varun Sundar Rabindranath <[email protected]> Signed-off-by: Bill Nell <[email protected]>
1 parent 250f1b7 commit 5318218

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/v1/attention/backends/mla/common.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -866,7 +866,7 @@ def forward(
866866

867867
if attn_metadata is None:
868868
# Profiling run.
869-
return output
869+
return output.fill_(0)
870870

871871
num_actual_toks = attn_metadata.num_actual_tokens
872872

0 commit comments

Comments
 (0)