Azure Cosmos DB (MongoDB): Metadata Throttling (429/3200) on startup

[thefunny1983]

What happened?

When using Azure Cosmos DB as the database backend, LibreChat encounters frequent rate-limiting errors on startup. The specific error is HTTP 429 TooManyRequests with a Cosmos DB substatus of 3200.

This indicates that the application is being throttled for making too many metadata requests (e.g., checking if a container exists), not for exceeding provisioned Request Units (RU/s). This suggests an issue with the database initialization pattern.

Root cause analysis
The root cause is not insufficient RUs but an application pattern that repeatedly makes metadata calls to Cosmos DB. The Cosmos DB SDK best practice is to perform initialization checks only once per application lifecycle.

The issue likely stems from the application code calling methods like CreateDatabaseIfNotExistsAsync or CreateContainerIfNotExistsAsync frequently, perhaps in a loop or on every request, instead of once at application startup.

Proposed Fix:
Refactor the Cosmos DB initialization logic to adhere to best practices:

1. Use a Singleton CosmosClient: Ensure a single instance of the CosmosClient is created and reused for the application's lifetime.

2. Initialize Once: Perform database and container existence checks (e.g., CreateContainerIfNotExistsAsync) only once when the application process starts.

3. Cache Resources: The resulting container and database objects should be cached in memory and reused for all subsequent operations.

This change will prevent the application from hitting the metadata rate limits and ensure a more stable and efficient integration with Azure Cosmos DB.

Version Information

ghcr.io/danny-avila/librechat-dev latest e9b3ab7eba0d 3 days ago 1.1GB
ghcr.io/danny-avila/librechat-rag-api-dev-lite latest 8858d2bd9ff5 12 days ago 1.48GB
ghcr.io/virtuos/librechat_exporter main 3012a75299b4 6 weeks ago 283MB

Steps to Reproduce

Configure LibreChat to use an Azure Cosmos DB for NoSQL endpoint.

Start the LibreChat application.

Observe the logs during the initialization phase.

What browsers are you seeing the problem on?

No response

Relevant log output

2025-07-23 10:05:27 error: [indexSync] error Error=16500, RetryAfterMs=500, Details='Response status code does not indicate success: TooManyRequests (429); Substatus: 3200; Reason: (
Errors : [
  "The request did not complete due to a high rate of metadata requests. Increasing request units will have no impact and is not recommended. Please retry the request. https://aka.ms/CosmosDB/sql/errors/metadata-429"
]
);
2025-07-23 10:05:27 error: Failed to update USER role permissions: Failed to retrieve or create role: Error=16500, RetryAfterMs=92, Details='Response status code does not indicate success: TooManyRequests (429); Substatus: 3200; Reason: (
Errors : [
  "The request did not complete due to a high rate of metadata requests. Increasing request units will have no impact and is not recommended. Please retry the request. https://aka.ms/CosmosDB/sql/errors/metadata-429"
]
);

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[thefunny1983]

This does not always happen though, only once in a while :-)

[danny-avila]

1. Is done, Initialization is only done once via singleton
2. Is controlled by mongoose, tailored to MongoDB. What will help here is additional configurations for the mongoose framework. There’s an open PR.
3. Already done

Since CosmosDB is not officially supported by the project (only mongodb), and it’s not easily stood up on my end, I would suggest you try some changes to the database handling, report back, and contribute if you’re open to it.