Skip to content

Commit a045492

Browse files
authored
vocab : call reserve() for building plamo-2-translate suffix (ggml-org#17343)
Test 'Q4_K_M' quantization on https://huggingface.co/pfnet/plamo-2-translate The 'suffix_to_score' size is 193510, it needs 19 memory allocation with final capacity 262144 to hold the value, if not preserve the memory. Signed-off-by: Haiyue Wang <[email protected]>
1 parent 1920345 commit a045492

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

src/llama-vocab.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1281,6 +1281,7 @@ struct llm_tokenizer_plamo2 : llm_tokenizer {
12811281

12821282
// Build suffix list in lexicographical order of reversed strings
12831283
std::vector<std::string> suffixes;
1284+
suffixes.reserve(suffix_to_score.size() + 1);
12841285
for (const auto & pair : suffix_to_score) {
12851286
suffixes.push_back(pair.first);
12861287
}

0 commit comments

Comments
 (0)