DeepSeek Api Stream Error, Both Models on Long Context input

[mesrashid]

What happened?

LibreChat | 2024-09-30 20:26:22 warn: [OpenAIClient.chatCompletion][stream] API error
Getting multiple similar errors everytime the context is increased current context tested >25000 tokens the error doesnot occur on small context lengths as long as the context in under 10k it streams..

Tried to get more detailed logs or error origins but couldn't trace. I suspect it aborts the request as I am not seeing any calls on my api when sending long context messages. Something in the abortController that is triggering this? Detailed specific logging improvements for custom api endpoints would be appreciated.

Steps to Reproduce

Single Prompt send a >25000 token message to the Deepseek models after configuring the base urls, the base urls tried are

1. /
2. /beta
3. /v1

What browsers are you seeing the problem on?

No response

Relevant log output

2024-09-30T20:28:50.306Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "https://api.deepseek.com/beta",
    modelOptions.model: "deepseek-chat",
    modelOptions.max_tokens: 50000,
    modelOptions.user: "66dd6b11f63397ca8b53ff41",
    modelOptions.stream: true,
    // 5 message(s)
    modelOptions.messages: [{"role":"user","content":"<requirements.txt>\nnpm install socket.io\nnpm install cassandra-driver\nn... [truncated],{"role":"assistant","content":"Certainly! Let's perform a detailed comparison between your existing ... [truncated],{"role":"user","content":"ok first lets get started with only Enhanced session recordings for mobile... [truncated],{"role":"assistant","content":"Certainly! Enhancing your session recording functionalities to better... [truncated],{"role":"user","content":"ok whats next"}],
}
2024-09-30T20:28:50.309Z debug: [OpenAIClient] chatCompletion: dropped params
{
  // 1 dropParam(s)
  dropParams: ["stop"],
    modelOptions.model: "deepseek-chat",
    modelOptions.max_tokens: 50000,
    modelOptions.user: "66dd6b11f63397ca8b53ff41",
    modelOptions.stream: true,
    // 5 message(s)
    modelOptions.messages: [{"role":"user","content":"<requirements.txt>\nnpm install socket.io\nnpm install cassandra-driver\nn... [truncated],{"role":"assistant","content":"Certainly! Let's perform a detailed comparison between your existing ... [truncated],{"role":"user","content":"ok first lets get started with only Enhanced session recordings for mobile... [truncated],{"role":"assistant","content":"Certainly! Enhancing your session recording functionalities to better... [truncated],{"role":"user","content":"ok whats next"}],
}
2024-09-30T20:28:50.313Z debug: Making request to https://api.deepseek.com/beta/chat/completions
2024-09-30T20:28:50.317Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2024-09-30T20:28:52.815Z warn: [OpenAIClient.chatCompletion][stream] API error
2024-09-30T20:28:52.816Z debug: [spendTokens] conversationId: edd00fb5-0611-4e0a-8487-758c852648e7 | Context: message | Token usage:
{
  promptTokens: 38592,
  completionTokens: 0,
}
2024-09-30T20:28:52.826Z debug: [spendTokens] No transactions incurred against balance
2024-09-30T20:28:52.830Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2024-09-30T20:28:52.839Z debug: [AskController] Request closed

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[mesrashid]

I have managed to debug and resolve the bug. I had to drop the 'stream' parameter in the librechat.yaml for my deepseek model.

custom:
# deepseek
# https://platform.deepseek.com/api_keys
# Model list: https://platform.deepseek.com/api-docs/pricing
- name: "deepseek"
apiKey: "${DEEPSEEK_API_KEY}"
baseURL: "https://api.deepseek.com/beta"
models:
default: [
"deepseek-chat",
"deepseek-coder"
]
fetch: false
titleConvo: true
titleModel: "deepseek-chat"
summarize: false
summaryModel: "deepseek-chat"
forcePrompt: false
dropParams: ['stop', 'stream'] //Like so
modelDisplayLabel: "DeepSeek"

And then retried the long contexts again, and this finally exposed the true error/ log from deepseek servers?

LibreChat | 2024-09-30 20:58:17 warn: [OpenAIClient.chatCompletion][create] API error
LibreChat | 2024-09-30 20:58:17 error: [OpenAIClient.chatCompletion] Unhandled error type Error: 400 Invalid max_tokens value, the valid range of max_tokens is [1, 8192]
LibreChat | 2024-09-30 20:58:17 error: [handleAbortError] AI response error; aborting request: Error: 400 Invalid max_tokens value, the valid range of max_tokens is [1, 8192]

I had set the output tokens to 50k hence a hard error on the api. The output tokens can only be between 1 and 8192, retried after setting it to 8192. And both without stream and with stream now work perfectly. Max output tokens was the key an incorrect entry here was causing the issue.

[danny-avila]

You are adjusting the max output tokens when you should be adjust max context tokens.

This is the one you want to adjust:

[mesrashid]

Yes, I usually set the context tokens to 75000 and output tokens to 50000 as I was using o1 models via open router. Just kept the numbers the same for every other model as well. The issue stemmed from exceeding 8192 tokens on the max output. The context tokens were within range. Actual input tokens were 25-35 thousand tokens.

[danny-avila]

AI response error; aborting request: Error: 400 Invalid max_tokens value, the valid range of max_tokens is [1, 8192]

This error comes from the AI provider (Openrouter if that's what you're using), there's nothing we can do here other than taking that into account on the provider/model basis

[mesrashid]

Yes that is perfect, it is just that the error was logged after dropping the stream parameter dropParams: ['stop', 'stream'], without this the error was very generic and not helpful. It just gave this line
2024-09-30T20:28:52.815Z warn: [OpenAIClient.chatCompletion][stream] API error

[chenxiaohui]

Yes ,I also encounterd this error.