Overloaded Errors Using Anthropic Models

[genaiservice]

What happened?

Hi team,

Thank you for your hard work and for providing the open-source project LibreChat to the community.

I would like to bring to your attention some errors I have encountered while using the Anthropic models. This issue is not urgent and does not require immediate resolution, but I wanted to share it for your awareness and for potential future improvements.

Thank you very much for your ongoing support and great work!

Best regards,

Version Information

Actual commit version:709922e5df2c7c1f4474a5bba6408f24294335aa

Steps to Reproduce

configured in my .env file as follows:

makefile:
ANTHROPIC_API_KEY=***
ANTHROPIC_MODELS=claude-sonnet-4-20250514,claude-3-7-sonnet-latest,claude-3-5-haiku-latest,claude-3-7-sonnet-20250219,claude-3-5-haiku-20241022,claude-3-5-sonnet-latest,claude-3-5-sonnet-20241022,claude-opus-4-20250514,claude-3-haiku-20240307

What browsers are you seeing the problem on?

No response

Relevant log output

For model claude-3-haiku-20240307:
2025-07-15 12:47:34 error: [api/server/controllers/agents/client.js #sendCompletion] Operation aborted 400 {"type":"error","error":{"type":"invalid_request_error","message":"max_tokens: 8192 > 4096, which is the maximum allowed number of output tokens for claude-3-haiku-20240307"}}
2025-07-15 12:47:34 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type 400 {"type":"error","error":{"type":"invalid_request_error","message":"max_tokens: 8192 > 4096, which is the maximum allowed number of output tokens for claude-3-haiku-20240307"}}

For model claude-opus-4-20250514:
2025-07-15 12:51:36 error: [api/server/controllers/agents/client.js #sendCompletion] Operation aborted {"type":"error","error":{"details":null,"type":"overloaded_error","message":"Overloaded"}}
2025-07-15 12:51:36 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type {"type":"error","error":{"details":null,"type":"overloaded_error","message":"Overloaded"}}

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[josegtmonteiro]

We are also getting the "Overloaded" error.

I understand that they are from Anthropic API: Anthropic API Errors - Overloaded

However, would it be possible to implement some retry logic (potentially with exponential delays) on the Librechat side to avoid the user receiving the error? For the Anthropic API, both 429 (rate limit) and 529 (overloaded) errors could have a retry mechanism on the Librechat side to prevent user-facing errors.

[harsh51191]

+1
And adding more.
If the error is wrt a specific provider's API it would be helpful to suggest to try other providers as most of the cases the task at hand can be done by others too.