Skip to content

Commit 3a37ae4

Browse files
author
ochafik
committed
comment / warn about preserved tokens not being single tokens
1 parent 5fd28b3 commit 3a37ae4

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

examples/server/server.cpp

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -383,6 +383,9 @@ struct server_task {
383383
if (ids.size() == 1) {
384384
LOG_DBG("Preserved token: %d\n", ids[0]);
385385
params.sampling.preserved_tokens.insert(ids[0]);
386+
} else {
387+
// This may happen when using a tool call style meant for a model with special tokens to preserve on a model without said tokens.
388+
LOG_DBG("Not preserved because more than 1 token: %s\n", t.get<std::string>().c_str());
386389
}
387390
}
388391
}

0 commit comments

Comments
 (0)