Add max_connections cache_params

Author: berri-teddy

docs/my-website/docs/proxy/caching.md

@@ -958,6 +958,19 @@ curl http://localhost:4000/v1/chat/completions \
 
 </Tabs>
 
+
+## Redis max_connections
+
+You can set the `max_connections` parameter in your `cache_params` for Redis. This is passed directly to the Redis client and controls the maximum number of simultaneous connections in the pool. If you see errors like `No connection available`, try increasing this value:
+
+```yaml
+litellm_settings:
+  cache: true
+  cache_params:
+    type: redis
+    max_connections: 100
+```
+
 ## Supported `cache_params` on proxy config.yaml
 
 ```yaml
@@ -966,6 +979,7 @@ cache_params:
   ttl: Optional[float]
   default_in_memory_ttl: Optional[float]
   default_in_redis_ttl: Optional[float]
+  max_connections: Optional[Int]
 
   # Type of cache (options: "local", "redis", "s3")
   type: s3

docs/my-website/docs/proxy/config_settings.md

@@ -50,6 +50,7 @@ litellm_settings:
     port: 6379  # The port number for the Redis cache. Required if type is "redis".
     password: "your_password"  # The password for the Redis cache. Required if type is "redis".
     namespace: "litellm.caching.caching" # namespace for redis cache
+    max_connections: 100  # [OPTIONAL] Maximum number of Redis connections. Passed directly to redis-py. Increase if you see 'No connection available' errors under high load.
 
     # Optional - Redis Cluster Settings
     redis_startup_nodes: [{"host": "127.0.0.1", "port": "7001"}] 

docs/my-website/docs/proxy/db_deadlocks.md

@@ -84,3 +84,29 @@ LiteLLM emits the following prometheus metrics to monitor the health/status of t
 | `litellm_in_memory_spend_update_queue_size`         | In-memory aggregate spend values for keys, users, teams, team members, etc.| In-Memory    |
 | `litellm_redis_spend_update_queue_size`             | Redis aggregate spend values for keys, users, teams, etc.                  | Redis        |
 
+
+## Troubleshooting: Redis Connection Errors
+
+You may see errors like:
+
+```
+LiteLLM Redis Caching: async async_increment() - Got exception from REDIS No connection available., Writing value=21
+LiteLLM Redis Caching: async set_cache_pipeline() - Got exception from REDIS No connection available., Writing value=None
+```
+ 
+This means all available Redis connections are in use, and LiteLLM cannot obtain a new connection from the pool. This can happen under high load or with many concurrent proxy requests.
+
+**Solution:**
+
+- Increase the `max_connections` parameter in your Redis config section in `proxy_config.yaml` to allow more simultaneous connections. For example:
+
+```yaml
+litellm_settings:
+  cache: True
+  cache_params:
+    type: redis
+    max_connections: 100  # Increase as needed for your traffic
+```
+
+Adjust this value based on your expected concurrency and Redis server capacity.
+

docs/my-website/docs/proxy/load_balancing.md

@@ -172,6 +172,9 @@ router_settings:
   redis_host: <your redis host>
   redis_password: <your redis password>
   redis_port: 1992
+  cache_params:
+    type: redis
+    max_connections: 100  # maximum Redis connections in the pool; tune based on expected concurrency/load
 ```
 
 ## Router settings on config - routing_strategy, model_group_alias