Getting "final prompt starts with 2 BOS tokens" warning with DeepSeek V3.1

[Lissanro]

App Version

3.26.2

API Provider

OpenAI Compatible

Model Used

DeepSeek V3.1

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

Load DeepSeek V3.1 with --jinja enabled in ik_llama.cpp, I get this warning on every prompt:

check_double_bos_eos: Added a BOS token to the prompt as specified by the model but the prompt also starts with a BOS token. So now the final prompt starts with 2 BOS tokens. Are you sure this is what you want?

Looks like Roo is adding BOS token in the prompt?

https://docs.unsloth.ai/basics/deepseek-v3.1-how-to-run-locally article mentions "A BOS is forcibly added ... For llama.cpp / GGUF inference, you should skip the BOS since it’ll auto add it."

💥 Outcome Summary

There should not be duplicate BOS token since this potentially can reduce quality of output by the model.

📄 Relevant Logs or Errors (Optional)

[roomote]

I'm exploring a potential fix for the duplicate BOS token issue with DeepSeek V3.1. I'll investigate how Roo Code handles BOS tokens for OpenAI Compatible providers and work on preventing the duplication.

[roomote]

I've opened PR #7501 with a fix for the duplicate BOS token issue: #7501

The solution adds a new configuration option openAiSkipSystemMessage that, when enabled for DeepSeek models used through OpenAI Compatible providers, merges the system prompt into the first user message instead of sending it as a separate system message. This prevents the duplicate BOS token warning you were experiencing with llama.cpp.

Most CI checks are passing, though there's one integration test failure that may need attention. The core functionality and all unit tests for the new feature are passing successfully.

[daniel-lxs]

After investigating this issue, the fundamental problem is that we cannot reliably detect if llama.cpp is being used at runtime from the API level. The OpenAI-compatible endpoint doesn't expose any headers or identifiers that would let us know it's llama.cpp with the --jinja flag enabled.

Without runtime detection, any solution would rely on guessing based on model names or requiring manual configuration, neither of which provides a robust fix. The duplicate BOS token warning is actually llama.cpp working as designed - it's adding BOS tokens because that's what the --jinja flag tells it to do.

The proper fix would need to be at the llama.cpp level, either by:

1. Not adding BOS tokens when they're already present in the input
2. Providing a way to disable auto-BOS via the API
3. Exposing runtime information so clients can adapt their behavior

For now, users experiencing this issue could:

• Run llama.cpp without the --jinja flag if their use case allows
• Use a different inference backend that doesn't auto-add BOS tokens
• Live with the warning (it's just a warning, not an error)