Skip to content

Fixes #15247 | Update chat.cpp to support (at least) qwen3 reasoning + tool_choice = required #15248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+7 −2
Open

Fixes #15247 | Update chat.cpp to support (at least) qwen3 reasoning + tool_choice = required #15248

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c6c4f7c
Update chat.cpp to support (at least) qwen3 + tool_choice = required
ExtReMLapin Aug 11, 2025
42937a5
refactored changes to follow string tern op
ExtReMLapin Aug 11, 2025
5796938
fixing editorconfig-checker CI (tailing whitespace)
ExtReMLapin Aug 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions common/chat.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1586,8 +1586,9 @@ static common_chat_params common_chat_params_init_hermes_2_pro(const common_chat
}

if (!inputs.tools.is_null()) {
auto supports_thinking = tmpl.source().find("<think>") != std::string::npos && data.thinking_forced_open == false;
// (content)?(<tool_call>{"name": "foo", "arguments": {"a": 1}}</tool_call>)*
data.grammar_lazy = inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_REQUIRED;
data.grammar_lazy = inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_REQUIRED || supports_thinking;
data.grammar = build_grammar([&](const common_grammar_builder & builder) {
std::vector<std::string> tool_rules;
std::vector<std::string> tool_call_alts;
Expand Down Expand Up @@ -1639,8 +1640,12 @@ static common_chat_params common_chat_params_init_hermes_2_pro(const common_chat
tool_call_alts.push_back(
"( \"```\\n\" | \"```json\\n\" | \"```xml\\n\" ) space " + wrappable_tool_call + " space \"```\" space ");
auto tool_call = builder.add_rule("tool_call", string_join(tool_call_alts, " | "));
if (supports_thinking) {
builder.add_rule("thinking", "\"<think>\" [^\\x00]* \"</think>\" space");
}
builder.add_rule("root",
std::string(data.thinking_forced_open ? "( \"</think>\" space )? " : "") +
std::string(supports_thinking ? "(thinking)? space " :
data.thinking_forced_open ? "( \"</think>\" space )? " : "") +
(inputs.parallel_tool_calls ? "(" + tool_call + ")+" : tool_call));
// Trigger on some common known "good bad" outputs (only from the start and with a json that's about a specific argument name to avoid false positives)
data.grammar_triggers.push_back({
Expand Down
Loading