Skip to content

Commit 5c3d948

Browse files
committed
added back accidental removal of clean spaces for hunyuan
1 parent 5856f8c commit 5c3d948

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

src/llama-vocab.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1669,6 +1669,7 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
16691669
} else if (
16701670
tokenizer_pre == "hunyuan") {
16711671
pre_type = LLAMA_VOCAB_PRE_TYPE_HUNYUAN;
1672+
clean_spaces = false;
16721673
} else if (
16731674
tokenizer_pre == "smoldocling") {
16741675
pre_type = LLAMA_VOCAB_PRE_TYPE_SMOLDOCLING;

0 commit comments

Comments
 (0)