Per-request clients vs single client for fastapi server

[lucabergamini]

Hello,
we are in the process of migrating from sync boto3 to aiobotocore. The main reason is that we have been bitten several time already by sync issues (most recently by performing too many actions in parallel in run_in_executor resulting in a thread shortage which has stalled following requests with a cascade effect on latency).

We currently use 3 different client types (kinesis, iot-core and s3) and we have around 100 endpoints being called several time per second. Our app is replicated with docker replicas and each replica has 1 CPU allocated right now.

In this setting, which option would be ideal?

1. one single session created at startup, one client per type per endpoint or when needed in the endpoint;
2. one single session and one single client per type in total

At first glance, 2 looks better if creating a client has an overhead, but according to the docs clients are mostly used as context managers when needed. However, in some previous issues it was listed as an option (#806). Could you please clarify on this point?

[thehesiod]

you want to keep as few clients as possible for as long as possible. each client has its own connection pool. also look into aioboto3 for a smoother transition

[lucabergamini]

Hi @thehesiod thanks for the reply.

| you want to keep as few clients as possible for as long as possible. each client has its own connection pool

So I guess the issue with a single one here would be exhausting the connection pool. Is there a setting to tune the pool size? Maybe I could just set that to a high value and use a single client. Otherwise it would be fairly complicated to find the correct number of clients required.

| also look into aioboto3 for a smoother transition
Any reason to prefer aioboto3 over this library? I tried to replace 1:1 our current APIs call with aiobotocore and for now nothing has break, the most complicated part is the context handling, but both libraries use that a far as I can tell

[thehesiod]

yes you can configure the pool size with the same way you do this with botocore, via the Config object. I only suggested aioboto3 since the apis are slightly different and it has some extra functionality, but yes it can all be done with aiobotocore

[lucabergamini]

Cool, will give it a try! Thanks a lot for the suggestion, please feel free to mark this as solved.