Add Kimi K2 Context Window

[njbbaer]

What happened?

UPDATE: The issue is that Max Context Tokens setting is being set far too low by default for the Kimi K2 model through OpenRouter. Manually setting it to the model's real max context size of 131000 fixed the issue. However, there is probably still a bug here related to how the maximum context size is determined for models through OpenRouter.

I've noticed issues attempting to use the Kimi K2 model through OpenRouter (moonshotai/kimi-k2) where some past details in the conversation appear to be very quickly forgotten by the model.

After some investigation I've found this is caused by LibreChat, not OpenRouter or the Kimi K2 model itself. By intercepting requests from LibreChat to OpenRouter using mitmproxy, I can see that past messages are sometimes dropped from the conversation history in the request sent to the OpenRouter completions endpoint.

I'm unsure of the exact parameters of this bug, but I can consistently reproduce it with the procedure described below. Having longer response in the conversation history appears to be necessary to trigger it. Kimi K2 is the only model I've found to be affected.

Let me know if you have difficulty reproducing this and I can investigate further.

Version Information

0.7.9-rc1

Steps to Reproduce

1. Start a new chat and select the model moonshotai/kimi-k2 via OpenRouter
2. In the first message, ask the model to remember something. (ex. Remember this fact: the apple is purple. Don't repeat it to me, just say you'll remember it.)
3. Then, ask the model to write a long response on any topic. (ex. Write a long essay on the Frutiger Aero aesthetic)
4. Finally, ask the model to recall the original fact. (ex. What color was the apple?)
5. See that the model does not know the answer.
6. Switch to any other model (OpenRouter or otherwise) and see that it does correctly recall the fact.
7. Intercept API requests to OpenRouter using something like mitmproxy and repeat the request to Kimi K2.
8. See that the intercepted request is missing the first user message.

For example, see the following intercepted request:

{
    "include_reasoning": true,
    "messages": [
        {
            "content": "## Welcome\n\nYour role is to provide expert advice...",
            "role": "system"
        },
        {
            "content": "I'll remember it.",
            "role": "assistant"
        },
        {
            "content": "Now write a long essay about the Frutiger Aero aesthetic.",
            "role": "user"
        },
        {
            "content": "Frutiger Aero emerged as a transitional visual language...",
            "role": "assistant"
        },
        {
            "content": "Do you remember what I asked you to remember? What color was the apple?",
            "role": "user"
        }
    ],
    "model": "moonshotai/kimi-k2",
    "stream": true,
    "temperature": 1,
}

What browsers are you seeing the problem on?

No response

Relevant log output

N/A

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Closed by #8604