Skip to content

Commit 5e6229a

Browse files
authored
common : fix double bos, use common_chat_templates for add_bos and add_eos (#15326)
This commit updates common_chat_templates_apply_jinja to use the the add_bos and add_eos parameters from the chat template instead of the inputs. The motivation for this is that currently if the `add_bos` and `add_eos` from the input parameters are used it is possible to there will be a missmatch between the model and the chat template which can lead to the the removal of duplicate BOS/EOS tokens in chat.cpp `apply` to not happen leading to two BOS tokens being added to the template.
1 parent e2c1bff commit 5e6229a

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

common/chat.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2061,8 +2061,8 @@ static common_chat_params common_chat_templates_apply_jinja(
20612061
params.enable_thinking = inputs.enable_thinking;
20622062
params.grammar = inputs.grammar;
20632063
params.now = inputs.now;
2064-
params.add_bos = inputs.add_bos;
2065-
params.add_eos = inputs.add_eos;
2064+
params.add_bos = tmpls->add_bos;
2065+
params.add_eos = tmpls->add_eos;
20662066

20672067
params.extra_context = json::object();
20682068
for (auto el : inputs.chat_template_kwargs) {

0 commit comments

Comments
 (0)