Skip to content

[BUG] thinking section not parsed for gpt-oss 120b #27

@avidwriter

Description

@avidwriter

OS

Linux

GPU Library

CUDA

YALS commit sha

34e09fb

Describe the bug

here is a sample output for gpt-oss 120b, it looks like it's not parsing the thinking section.

<|channel|>analysis<|message|>The user just says "a joke". They want a joke. We should respond with a joke. Could be any style. Probably a short, witty joke. Could ask what kind, but they just said "a joke". So deliver a joke. Maybe a classic. Also be mindful of being appropriate. Provide a light-hearted joke. Let's give a short joke.<|end|><|start|>assistant<|channel|>final<|message|>Why don’t scientists trust atoms anymore?

Because they make up everything! 😄

Reproduction steps

using openwebui frontend
YALS backend
prompt: tell a joke

Expected behavior

should have a reasoning block before the answer

Logs

No response

Additional context

I think llama.cpp fixed this, might just need to update the deps.

Thanks!

Acknowledgements

  • I have looked for similar issues before submitting this one.
  • I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
  • I understand that the developers have lives and my issue will be answered when possible.
  • I understand the developers of this program are human, and I will ask my questions politely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions