-
-
Notifications
You must be signed in to change notification settings - Fork 9
[BUG] thinking section not parsed for gpt-oss 120b #27
Description
OS
Linux
GPU Library
CUDA
YALS commit sha
Describe the bug
here is a sample output for gpt-oss 120b, it looks like it's not parsing the thinking section.
<|channel|>analysis<|message|>The user just says "a joke". They want a joke. We should respond with a joke. Could be any style. Probably a short, witty joke. Could ask what kind, but they just said "a joke". So deliver a joke. Maybe a classic. Also be mindful of being appropriate. Provide a light-hearted joke. Let's give a short joke.<|end|><|start|>assistant<|channel|>final<|message|>Why don’t scientists trust atoms anymore?
Because they make up everything! 😄
Reproduction steps
using openwebui frontend
YALS backend
prompt: tell a joke
Expected behavior
should have a reasoning block before the answer
Logs
No response
Additional context
I think llama.cpp fixed this, might just need to update the deps.
Thanks!
Acknowledgements
- I have looked for similar issues before submitting this one.
- I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
- I understand that the developers have lives and my issue will be answered when possible.
- I understand the developers of this program are human, and I will ask my questions politely.