common : ensure llama_batch size does not exceed max size (#9668)

A crash was observed when the number of tokens added to a batch exceeds llama_batch size. An assertion in llama_batch_add was added to protect against llama_batch size overflow.
2024-12-24 02:14:35 +00:00 · 2024-09-29 05:25:00 -07:00 · 2024-09-29 05:25:00 -07:00 · faac0bae26
commit faac0bae26
parent f99d3f8367
1 changed files with 2 additions and 0 deletions
--- a/common/common.cpp
+++ b/common/common.cpp
@ -1437,6 +1437,8 @@ void llama_batch_add(
                          llama_pos   pos,
    const std::vector<llama_seq_id> & seq_ids,
                               bool   logits) {
+    GGML_ASSERT(batch.seq_id[batch.n_tokens] && "llama_batch size exceeded");
+
    batch.token   [batch.n_tokens] = id;
    batch.pos     [batch.n_tokens] = pos;
    batch.n_seq_id[batch.n_tokens] = seq_ids.size();