Skip to content

Commit c197f0f

Browse files
committed
common: ensure token addition to batch does not exceed llama_batch size
A crash was observed when the number of tokens added to a batch exceeds llama_batch size. An assertion in llama_batch_add was added to protect against llama_batch size overflow.
1 parent 95bc82f commit c197f0f

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

common/common.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1432,6 +1432,8 @@ void llama_batch_add(
14321432
llama_pos pos,
14331433
const std::vector<llama_seq_id> & seq_ids,
14341434
bool logits) {
1435+
GGML_ASSERT(batch.seq_id[batch.n_tokens] && "llama_batch size exceeded");
1436+
14351437
batch.token [batch.n_tokens] = id;
14361438
batch.pos [batch.n_tokens] = pos;
14371439
batch.n_seq_id[batch.n_tokens] = seq_ids.size();

0 commit comments

Comments
 (0)