Add new kimi model to groq #7692
Conversation
dosubot
bot
added
size:S
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| | "openai/gpt-oss-20b" | ||
|
|
||
| export const groqDefaultModelId: GroqModelId = "llama-3.3-70b-versatile" // Defaulting to Llama3 70B Versatile | ||
| export const groqDefaultModelId: GroqModelId = "moonshotai/kimi-k2-instruct-0905" |
There was a problem hiding this comment.
llama-3.3-70b-versatile to moonshotai/kimi-k2-instruct-0905 will affect all users who rely on the default model. They may experience different behavior, costs, and capabilities. Could we consider:
- Keeping the existing default for backward compatibility
- Or documenting this breaking change prominently in the changelog/release notes?
| contextWindow: 262144, | ||
| supportsImages: false, | ||
| supportsPromptCache: true, | ||
| inputPrice: 0.6, |
There was a problem hiding this comment.
Is the significant price reduction intentional? The new model has:
- Input price: /bin/sh.6 (40% lower than the existing kimi-k2-instruct at .0)
- Output price: .5 (17% lower than the existing kimi-k2-instruct at .0)
Could you confirm these prices are correct according to Groq's pricing?
| supportsPromptCache: true, | ||
| inputPrice: 0.6, | ||
| outputPrice: 2.5, | ||
| cacheReadsPrice: 0.15, |
There was a problem hiding this comment.
The cache reads price of /bin/sh.15 represents a 75% discount from the input price (/bin/sh.6), while the existing kimi-k2-instruct model has a 50% discount. Is this aggressive discount structure correct?
| inputPrice: 0.6, | ||
| outputPrice: 2.5, | ||
| cacheReadsPrice: 0.15, | ||
| description: "Moonshot AI Kimi K2 Instruct 1T model, 256K context.", |
There was a problem hiding this comment.
Minor inconsistency: The description says "256K context" but the contextWindow value is 262144 (which is 256Ki or 256 * 1024). While technically correct, this differs from the pattern used in the existing model which says "128K context" with value 131072. Should we use "256Ki context" or keep it consistent with the existing pattern?
hannesrudolph
added
the
Issue/PR - Triage
Important
Add
moonshotai/kimi-k2-instruct-0905model togroq.tsand set it as the default model with specific configurations.moonshotai/kimi-k2-instruct-0905toGroqModelIdandgroqModelsingroq.ts.groqDefaultModelIdtomoonshotai/kimi-k2-instruct-0905ingroq.ts.moonshotai/kimi-k2-instruct-0905hasmaxTokens: 16384,contextWindow: 262144,supportsImages: false,supportsPromptCache: true,inputPrice: 0.6,outputPrice: 2.5,cacheReadsPrice: 0.15.This description was created by
for c43f39e. You can customize this summary. It will automatically update as commits are pushed.