Doesn't handle `thinking` key for ollama/gpt-oss responses

[kovach]

Please update gptel first -- errors are often fixed by the time they're reported.

• I have updated gptel to the latest commit and tested that the issue still exists

Bug Description

While using

• ollama 0.11.2
• gptel 20250808.939
• model: gpt-oss:20b

gptel does not include any thinking output.

The reason for this seems to be that the API responses for this model have this shape: ... "message":{"role":"assistant","content":"","thinking":"..."}...,
unlike the others I tried which only use content and bracket their thinking with a special word. Now, every message either has a nonempty content or thinking field but not both.

Backend

Ollama

Steps to Reproduce

Using a gpt-oss model through ollama, attempt any request with "Include reasoning" enabled. No reasoning steps are included in output.

Additional Context

• ollama 0.11.2
• gptel 20250808.939
• model: gpt-oss:20b

Backtrace

Log Information

[karthink]

Thanks for the report.

I can't run the 20B model on Ollama as I don't have the hardware for it. Please generate a log of the response that I can refer to. The logs will be different for streaming and non-streaming responses, I need both. You can turn on/off streaming via gptel-stream.

Instructions for generating a log are in the bug report template.

[bjodah]

I think I encountered this as well. I'm running llama.cpp with gpt-oss-120b, and gptel does not identify the reasoning content, which is a problem for subsequent requests since special tokens are then part of the prompt.

Here's a sample gptel session:

### Writing elisp, can you help?                                                                                                                                                                                                                                              
```elisp                                                                                                                                                                                                                                                                      
(defun bar()                                                                                                                                                                                                                                                                  
  "Hello \"World\"!" ; <--- I don't want to return this string, instead I want it inserted wherever the user has its cursor                                                                                                                                                   
  )                                                                                                                                                                                                                                                                           
```

attaching logs for both streaming and non-streaming.

gptel-nonstreaming.log
gptel.log

EDIT: the error you get currently if you don't manually remove the reasoning content looks as follows:

my-local-llm-backend error: ((HTTP/1.1 500 Internal Server Error) server_error) You have passed a message containing <|channel|> tags in the content field. Instead of doing this, you should pass analysis messages (the string between '<|message|>' and '<|end|>') in the 'th\
inking' field, and final messages (the string between '<|message|>' and '<|end|>') in the 'content' field. at row 271, column 36: 

[karthink]

@bjodah Thanks. This is the first time I'm seeing this response format. Is Ollama's thinking format documented anywhere? In the official documentation I only see a reference to "thinking" in the message format for what the client (gptel) should send, not what Ollama sends back.

This is the first time I'm seeing <|channel|> and <|message|> tags in the content of an Ollama response, and none of the other reasoning models Ollama supports do this.

[kovach]

here are the logs for ollama:

gptel-nonstreaming.log
gptel-streaming.log

I think @bjodah was referring to llama.cpp, not ollama. I switched over to llama.cpp myself, and although I'm seeing the same tags in the model response that they mention, I'm not having any issues with backend errors (I built llama.cpp from source yesterday). gptel is working great, albeit with the those tags showing up in the output (which I take it is not intended?).

[bjodah]

@karthink sorry, I should have paid more attention to this issue being about ollama, specifically. I was running llama.cpp with --reasoning-format none. I now notice that since about one day there have been changes to llama.cpp with respect to gpt-oss so one no longer needs to run with that flag. I'll recompile llama.cpp and try to see how it fares without --reasoning-format none.

[bjodah]

with a freshly built llama.cpp, and using gpt-oss in streaming mode from gptel, the answer is always prefixed by:
"<|channel|>analysis", do you see this as well @kovach ?

[karthink]

I'm a little confused about this issue.

@kovach, @bjodah Is this an issue with Ollama or Llama.cpp?

[bjodah]

@karthink I just recompiled latest master of llama.cpp. The earlier problem with <|channel|>analysis leaking into response is now gone. Since llama.cpp's openai v1 compatible endpoint can now split the model output into reasoning_content and content I think there's no need for gptel to directly parsing the underlying harmony format. Thank you for your continued work on gptel, it's a true gem!

[karthink]

I'm not sure if the ollama issue is fixed. Since @kovach isn't using it anymore, I'm closing this issue for now.

You can reopen or create a new one if it's still not working.