Skip to content

Commit 0373486

Browse files
authored
graph : fix assert in memory-less build_attn (ggml-org#15590)
ggml-ci
1 parent 62cef26 commit 0373486

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/llama-graph.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1376,7 +1376,7 @@ ggml_tensor * llm_graph_context::build_attn(
13761376

13771377
// [TAG_NO_CACHE_PAD]
13781378
// TODO: if ubatch.equal_seqs() == true, we can split the three tensors below into ubatch.n_seqs_unq streams
1379-
assert(!ubatch.equal_seqs());
1379+
assert(!ubatch.equal_seqs() || (k_cur->ne[3] == 1 && k_cur->ne[3] == ubatch.n_seqs_unq));
13801380

13811381
ggml_tensor * q = q_cur;
13821382
ggml_tensor * k = k_cur;

0 commit comments

Comments
 (0)