OpenAI Compatible with llama.cpp not working

[laurentvv]

App Version

Version: 3.25.14

API Provider

OpenAI Compatible

Model Used

Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf

Roo Code Task Links (Optional)

http://127.0.0.1:8080
Api key : None
Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf

🔁 Steps to Reproduce

Windows 11

LLAMA command :
llama-server.exe -m Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf -c 16000

Roo Code config :
Config OpenAI Compatible with
http://127.0.0.1:8080
Api key : None
Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf

Ask or another mode : Invalid URL

Works with Cline or Kilo Code

💥 Outcome Summary

Expected OpenAI Compatible with llama.cpp to work, but got Invalid URL

📄 Relevant Logs or Errors (Optional)

[roomote]

I'm here to help out the maintainers and am going to see if I can fix the issue with OpenAI Compatible mode not working with llama.cpp. I'll investigate the Invalid URL error and work on a solution.

[roomote]

I've created a pull request to fix this issue: #7066

The problem was that when using OpenAI Compatible mode with llama.cpp, the OpenAI SDK expects the base URL to include the /v1 path component for API compatibility. The fix automatically appends /v1 to base URLs that are plain URLs without a path component.

With this change, you'll be able to use llama.cpp with Roo Code by simply providing http://127.0.0.1:8080 as the base URL, and it will automatically work with the OpenAI-compatible endpoints.

The PR includes tests to verify the URL handling works correctly for various URL formats. Most CI checks have passed - there's one unrelated integration test failure in a symlink test that appears to be a flaky test.

[nimishchaudhari]

I can confirm, I am facing this same issue. I thought I made something wrong in my configuration given I recently got myself a RTX 3090 and started riding the wave.

[daniel-lxs]

Hey @laurentvv Can you share the exact error message you are seeing or a screenshot of it?