Increase repository_azure max. threads on serverless #128130
Conversation
On Serverless, the `repository_azure` thread pool is shared between snapshots and translogs/segments upload logic. Because snapshots can be rate-limited when executing in the repository_azure thread pool, we want to leave enough room for the other upload threads to be executed. Relates ES-11391
tlrx
added
>non-issue
:Distributed Coordination/Snapshot/Restore
elasticsearchmachine
added
the
Team:Distributed Coordination
|
Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination) |
| public static ExecutorBuilder<?> executorBuilder() { | ||
| return new ScalingExecutorBuilder(REPOSITORY_THREAD_POOL_NAME, 0, 5, TimeValue.timeValueSeconds(30L), false); | ||
| public static ExecutorBuilder<?> executorBuilder(Settings settings) { | ||
| int repositoryAzureMax = 5; |
There was a problem hiding this comment.
I guess that we are good with 5 threads for segments/translog uploads?
There was a problem hiding this comment.
I think so, but we'll have to monitor production to know if that's enough I think.
henningandersen
left a comment
henningandersen
left a comment
There was a problem hiding this comment.
LGTM, great find.
I wonder if we should make what is conditioned by stateless now the default even? It seems snapshot meta uploads while doing X snapshot uploads could benefit from the same. But we can do that in a follow-up if this turns out successful.
|
Thanks Francisco and Henning. The ticket resolved by this change is ES-11734.
Yes, that makes sense too. I'll follow up on that. |
4 participants
On Serverless, the
repository_azurethread pool is shared between snapshots and translogs/segments upload logic. Because snapshots can be rate-limited when executing in the repository_azure thread pool, we want to leave enough room for the other upload threads to be executed.Relates ES-11391
Relates ES-11734