Skip to content

Commit fba5c0d

Browse files
authored
chat : hotfix gpt-oss jinja raising an exception (#15243)
* chat : hotfix gpt-oss jinja raising an exception * fix
1 parent 53d0a12 commit fba5c0d

File tree

1 file changed

+11
-0
lines changed

1 file changed

+11
-0
lines changed

common/chat.cpp

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -552,6 +552,17 @@ common_chat_templates_ptr common_chat_templates_init(
552552
default_template_src = CHATML_TEMPLATE_SRC;
553553
}
554554
}
555+
556+
// TODO @ngxson : this is a temporary hack to prevent chat template from throwing an error
557+
// Ref: https://github.com/ggml-org/llama.cpp/pull/15230#issuecomment-3173959633
558+
if (default_template_src.find("<|channel|>") != std::string::npos
559+
// search for the error message and patch it
560+
&& default_template_src.find("in message.content or") != std::string::npos) {
561+
string_replace_all(default_template_src,
562+
"{%- if \"<|channel|>analysis<|message|>\" in message.content or \"<|channel|>final<|message|>\" in message.content %}",
563+
"{%- if false %}");
564+
}
565+
555566
std::string token_bos = bos_token_override;
556567
std::string token_eos = eos_token_override;
557568
bool add_bos = false;

0 commit comments

Comments
 (0)