moonshotai/kimi-k2-instruct model issue

[DareDevil2000]

App Version

3.23.11

API Provider

Groq

Model Used

moonshotai/kimi-k2-instruct

Roo Code Task Links (Optional)

No response

🔁 Steps to Reproduce

use api from groq with architecture mode

💥 Outcome Summary

400 max_tokens must be less than or equal to 16384, the maximum value for max_tokens is less than the context_window for this model

📄 Relevant Logs or Errors (Optional)

[roomote]

I am working on fixing this issue. The problem is that the max_tokens parameter for the moonshotai/kimi-k2-instruct model is being set above its maximum limit of 16384. I will investigate the model configuration and implement a fix to respect this limit.

[dosubot]

The 400 error happens because the max_tokens value in your request is set higher than the allowed limit for the moonshotai/kimi-k2-instruct model on Groq. The error message indicates the maximum for max_tokens is 16,384, which is less than the model's full context window (the total of input + output tokens). This is a common restriction: max_tokens (the output length) must be less than or equal to the provider's limit, and the sum of your prompt tokens plus max_tokens must not exceed the model's context window.

Roo Code lets you configure max_tokens and context window for OpenAI-compatible providers like Groq in the provider settings. To resolve this, set max_tokens to 16,384 or less for this model in your Roo Code Groq provider configuration. If you need help finding or adjusting this setting, let me know and I can walk you through it.

I couldn't find official documentation listing the exact context window for kimi-k2-instruct on Groq, so if you continue to see issues, double-check Groq's latest model documentation or release notes for any recent changes to token limits.
Reference

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[roomote]

✅ Issue Fixed

I have successfully resolved the moonshotai/kimi-k2-instruct max_tokens limit issue.

🔧 Changes Made

1. Updated model configuration: Changed moonshotai/kimi-k2-instruct maxTokens from 8192 to 16384 to match the actual Groq API limit
2. Added validation: Enhanced BaseOpenAiCompatibleProvider to ensure max_tokens never exceeds model limits, even when users override with modelMaxTokens
3. Fixed TypeScript types: Added proper type checking to handle nullish values

🎯 Problem Solved

• ✅ Prevents 400 errors: "max_tokens must be less than or equal to 16384"
• ✅ User-configured modelMaxTokens values are now capped at model API limits
• ✅ Maintains backward compatibility while preventing API errors
• ✅ All tests pass and TypeScript compilation successful

📋 Pull Request

The fix has been implemented in PR #5730: #5730

The CI checks are currently running. Once they pass, the PR will be ready for review and merge.

You should no longer encounter the "max_tokens must be less than or equal to 16384" error when using the moonshotai/kimi-k2-instruct model with Groq API in architecture mode.

[mrubens]

This should be fixed in 3.23.12 - reopen if not. Thanks!