Skip to content

Commit b8498bc

Browse files
authored
re-enable custom paged attention for V0 (ROCm#511)
Signed-off-by: charlifu <[email protected]>
1 parent f68829f commit b8498bc

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm/platforms/rocm.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,8 @@ def use_rocm_custom_paged_attention(qtype: torch.dtype, head_size: int,
113113

114114
# rocm custom page attention not support on navi (gfx1*)
115115
return (ON_MI250_MI300 and not ON_NAVI
116-
and (sliding_window == 0 or sliding_window == (-1, -1))
116+
and (not envs.VLLM_USE_V1 or sliding_window == 0
117+
or sliding_window == (-1, -1))
117118
and (qtype == torch.half or qtype == torch.bfloat16)
118119
and (head_size == 64 or head_size == 128)
119120
and (block_size == 16 or block_size == 32)

0 commit comments

Comments
 (0)