Eval bug: Jinja parser not working with QwQ-32B

[Edremon]

Name and Version

version: 4842 (3d652bf)
built with cc (GCC) 14.2.1 20250207 for x86_64-pc-linux-gnu

Operating systems

Linux

GGML backends

CPU

Hardware

Irrelevant

Models

bartowski/Qwen_QwQ-32B-GGUF

Problem description & steps to reproduce

I would like to use --jinja to have the thinking separated. But, it doesn't work with that model. I initially thought that it was a problem with that specific GGUF, but, I tried copying the official jinja template, and I'm getting the same error. The jinja template is extremely similar to the included one Qwen-Qwen2.5-7B-Instruct.jinja (which work), the differences are:

diff --git a/Qwen-Qwen2.5-7B-Instruct.jinja b/QwQ-official.jinja
index bdf7919a..8844e349 100644
--- a/Qwen-Qwen2.5-7B-Instruct.jinja
+++ b/QwQ-official.jinja
@@ -3,7 +3,7 @@
     {%- if messages[0]['role'] == 'system' %}
         {{- messages[0]['content'] }}
     {%- else %}
-        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+        {{- '' }}
     {%- endif %}
     {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
     {%- for tool in tools %}
@@ -14,14 +14,16 @@
 {%- else %}
     {%- if messages[0]['role'] == 'system' %}
         {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
-    {%- else %}
-        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
-    {%- endif %}
+  {%- endif %}
 {%- endif %}
 {%- for message in messages %}
-    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
         {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" and not message.tool_calls %}
+        {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
     {%- elif message.role == "assistant" %}
+        {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
         {{- '<|im_start|>' + message.role }}
         {%- if message.content %}
             {{- '\n' + message.content }}
@@ -50,5 +52,5 @@
     {%- endif %}
 {%- endfor %}
 {%- if add_generation_prompt %}
-    {{- '<|im_start|>assistant\n' }}
+    {{- '<|im_start|>assistant\n<think>\n' }}
 {%- endif %}

I have tried touching it a bit, I initially thought that the lstrip weren't supported, since no others included template use it but, even removing those, I get the same error.

First Bad Commit

No response

Relevant log output

srv    load_model: load_model: The chat template that comes with this model is not yet supported, falling back to chatml. This may cause the model to output suboptimal responses

[CISC]

I think the problem is that minja doesn't support str.split (nor str.lstrip I guess).

cc @ochafik

[Edremon]

The R1's jinja template have set content = content.split('</think>')[-1] in them, I haven't tested if they work, but why would they be included if they don't?

[CISC]

I guess it would be easier to tell what the problem was if server also output the original runtime error...

[ochafik]

@Edremon thanks for the report & @CISC for the ping :-)

Adding the missing str methods to Minja in google/minja#56