Skip to content

Commit cd652ac

Browse files
authored
[BugFix] Fix kv_no_split not contiguous (#3711)
allgather need contiguous data, split operation return uncontiguous data. - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 Signed-off-by: zhaozx-cn <[email protected]>
1 parent 28a1529 commit cd652ac

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

vllm_ascend/attention/mla_v1.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1166,6 +1166,8 @@ def _mla_preprocess(self, layer_name, hidden_states, kv_cache,
11661166
dim=-1,
11671167
)
11681168
q_c = self.q_a_layernorm(q_c)
1169+
# allgather need contiguous data
1170+
kv_no_split = kv_no_split.contiguous()
11691171
else:
11701172
q_c = hidden_states
11711173
kv_no_split = self.kv_a_proj_with_mqa(hidden_states)[0]

0 commit comments

Comments
 (0)