Increase boto3 max_pool_connections from default 10 to 50

Prudhvi0717 · Prudhvi0717 · commit 55b68da44a17 · 2026-03-04T17:24:26.000+05:30
The default urllib3 connection pool size of 10 causes HTTP connection
queuing under concurrent LLM requests, degrading total response time
by ~60% at 20 concurrent requests.
diff --git a/singlestoredb/ai/chat.py b/singlestoredb/ai/chat.py
@@ -114,6 +114,7 @@ def obo_token_getter_fn() -> Optional[str]:
         cfg_kwargs = {
             'signature_version': UNSIGNED,
             'retries': {'max_attempts': 1, 'mode': 'standard'},
+            'max_pool_connections': 50,
         }
         if read_timeout is not None:
             cfg_kwargs['read_timeout'] = read_timeout

Original file line number	Diff line number	Diff line change
`@@ -114,6 +114,7 @@ def obo_token_getter_fn() -> Optional[str]:`
`114`	`114`	`cfg_kwargs = {`
`115`	`115`	`'signature_version': UNSIGNED,`
`116`	`116`	`'retries': {'max_attempts': 1, 'mode': 'standard'},`
	`117`	`+ 'max_pool_connections': 50,`
`117`	`118`	`}`
`118`	`119`	`if read_timeout is not None:`
`119`	`120`	`cfg_kwargs['read_timeout'] = read_timeout`