why io/timer poll only try wake one idle-workers. #5847
-
|
if a worker polled 128 tasks to localqueue, it only wake one worker, is this reasonable? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
We don't want to wake all sleeping workers at once due to thundering herd issues, however there's an interesting question here about how many to wake, as this might vary by workload and system configuration. With the current scheme, worker wakes will form a chain and eventually wake all workers given a suitable workload, however there's a question here about how quickly we should wake workers. We could wake a couple of workers instead of just one, but I'm not entirely certain what impact that would have. I suspect it would help workloads that are more cpu-bound, but I'm not sure what its impact would be on other systems, especially ones with high core counts. CC @carllerche |
Beta Was this translation helpful? Give feedback.
We don't want to wake all sleeping workers at once due to thundering herd issues, however there's an interesting question here about how many to wake, as this might vary by workload and system configuration.
With the current scheme, worker wakes will form a chain and eventually wake all workers given a suitable workload, however there's a question here about how quickly we should wake workers. We could wake a couple of workers instead of just one, but I'm not entirely certain what impact that would have.
I suspect it would help workloads that are more cpu-bound, but I'm not sure what its impact would be on other systems, especially ones with high core counts.
CC @carllerche