Skip to content

Commit 4d7c1d5

Browse files
authored
[Bugfix] Fix MRoPE dispatch on XPU (vllm-project#24724)
Signed-off-by: Yan Ma <[email protected]>
1 parent 41f17bf commit 4d7c1d5

File tree

1 file changed

+9
-0
lines changed
  • vllm/model_executor/layers/rotary_embedding

1 file changed

+9
-0
lines changed

vllm/model_executor/layers/rotary_embedding/mrope.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -300,6 +300,15 @@ def forward_cuda(
300300
key = torch.cat((key_rot, key_pass), dim=-1).reshape(key_shape)
301301
return query, key
302302

303+
def forward_xpu(
304+
self,
305+
positions: torch.Tensor,
306+
query: torch.Tensor,
307+
key: Optional[torch.Tensor] = None,
308+
offsets: Optional[torch.Tensor] = None,
309+
) -> tuple[torch.Tensor, Optional[torch.Tensor]]:
310+
return self.forward_native(positions, query, key, offsets)
311+
303312
def forward_cpu(
304313
self,
305314
positions: torch.Tensor,

0 commit comments

Comments
 (0)