[BUG] Roocode errors out with Kwaipilot / KAT-Dev Q8

[akierum]

Problem (one or two sentences)

Roocode errors out with Kwaipilot / KAT-Dev

I use Q8 version DevQuasar Kwaipilot.KAT-Dev.Q8_0.gguf

Context (who is affected and when)

Open vsCode, Kilocode.
Load model Kwaipilot.KAT-Dev.Q8_0.gguf
Use Jan.ai v0.7.1 or LMstudio 0.3.30

Reproduction steps

Win 11
2x RTX 3090
Kwaipilot.KAT-Dev.Q8_0.gguf
512GB RAM

Expected result

Work ok

Actual result

Roocode errors out have to restart

Variations tried (optional)

No response

App Version

3.28.15

API Provider (optional)

None

Model Used (optional)

Kwaipilot.KAT-Dev.Q8_0.gguf

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[roomote]

I'm exploring a potential fix for the Kwaipilot/KAT-Dev Q8 model compatibility issue and will open a PR to share progress.

[roo-code-cte]

Acknowledging this report. Investigating the Kwaipilot KAT-Dev Q8 model compatibility with Roo Code 3.28.15 using Jan.ai 0.7.1 and LM Studio 0.3.30. I will attempt a fix and open a PR to share progress. If possible, please share any Roo Code output panel logs and the exact action that triggers the error so I can reproduce more reliably.

[roomote]

I've opened PR #8576 with improved error handling for LM Studio model compatibility issues: #8576

The changes include:

• Better error detection for connection failures
• Clear messages when models are not found
• Context length error handling with actionable advice
• Comprehensive test coverage

All CI checks have passed. While this improves the error handling significantly, testing with the actual Kwaipilot/KAT-Dev model would help confirm if this fully resolves the compatibility issue. Feedback welcome!

[akierum]

Errors happen using Jan.ai too

[akierum]

Make sure you use Q8 version DevQuasar Kwaipilot.KAT-Dev.Q8_0.gguf
Do not use the mradermacher version because it does now work.

This is mradermacher Cline but same errors with RooCode too, so use the DevQuasar

[akierum]

Error
Unexpected API Response: The language model did not provide any assistant messages. This may indicate an issue with the API or the model's output.

OpenAI completion error: 502 Proxy request to model failed: error sending request for url (http://127.0.0.1:3227/chat/completions): error trying to connect: tcp connect error: No connection could be made because the target machine actively refused it. (os error 10061)

Retry attempt 1
Retrying now...

crash at 40k tokens. that is reproducible 100%