Skip to content

Commit d063cf1

Browse files
committed
common: ensure token addition to batch does not exceed llama_batch size
A crash was observed when the number of tokens added to a batch exceeds llama_batch size. An assertion in llama_batch_add was added to protect against llama_batch size overflow.
1 parent a0a7df6 commit d063cf1

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

common/common.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1437,6 +1437,8 @@ void llama_batch_add(
14371437
llama_pos pos,
14381438
const std::vector<llama_seq_id> & seq_ids,
14391439
bool logits) {
1440+
GGML_ASSERT(batch.seq_id[batch.n_tokens] && "llama_batch size exceeded");
1441+
14401442
batch.token [batch.n_tokens] = id;
14411443
batch.pos [batch.n_tokens] = pos;
14421444
batch.n_seq_id[batch.n_tokens] = seq_ids.size();

0 commit comments

Comments
 (0)