roocode with groq gives max_tokens error for kimi k2

[hamaadtahiir]

App Version

3.23.11

API Provider

Groq

Model Used

moonshotai/kimi-k2-instruct

Roo Code Task Links (Optional)

Roocode assumes max token for the kimi k2 through groq to be same as its context window, thus giving this error:

400 max_tokens must be less than or equal to 16384, the maximum value for max_tokens is less than the context_window for this model

🔁 Steps to Reproduce

Just try kimi k2 model through groq

💥 Outcome Summary

Expected it to work but it didnt

📄 Relevant Logs or Errors (Optional)

[dosubot]

This error happens because Roo Code sets max_tokens to the model's full context window (128K), but Groq's API for kimi k2 only allows max_tokens up to 16,384—much less than the context window. The context window is the total number of tokens (input + output) the model can process, while max_tokens is just the maximum output length. Setting max_tokens too high triggers the 400 error.

Roo Code has added more granular context and max_tokens controls for OpenAI-compatible providers and Groq models in recent versions, but there’s no specific documentation or workaround for this kimi k2/Groq mismatch yet. The best workaround is to manually set max_tokens to 16,384 or less for this model, if Roo Code’s UI or config allows it. If not, this is a current limitation and worth watching for updates or fixes in future releases. More details in the update notes.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[roomote]

I am working on fixing this issue. The problem is that Roo Code is setting max_tokens to the full context window (128K) for the kimi k2 model through Groq, but Groq only allows max_tokens up to 16,384 for this model. I will investigate the codebase to find where model configurations are handled and implement a fix to properly limit max_tokens for this specific model.

[roomote]

✅ Issue Fixed

I have successfully identified and fixed the issue with the moonshotai/kimi-k2-instruct model on Groq.

Problem Analysis

The error occurred because the model configuration had maxTokens set to 8192, but Groq API allows up to 16,384 tokens for this specific model. When Roo Code attempted to use the model, it was hitting the Groq API limit.

Solution Implemented

• Updated the maxTokens value for moonshotai/kimi-k2-instruct from 8192 to 16384 in the Groq provider configuration
• This aligns with Groq API limits and resolves the "max_tokens must be less than or equal to 16384" error

Pull Request

📋 PR #5740: Fix max_tokens limit for moonshotai/kimi-k2-instruct on Groq

Testing

• ✅ All existing Groq provider tests pass
• ✅ Type checking passes
• ✅ Linting passes
• ✅ CI checks are running

The fix is now ready for review and should resolve the issue once merged. Users will be able to use the moonshotai/kimi-k2-instruct model through Groq without encountering the max_tokens error.

[mrubens]

This is already fixed in 3.23.12, released a few minutes ago! Try it out and reopen if still an issue. Thanks!