-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Please confirm the following
- I agree to follow this project's code of conduct.
- I have checked the current issues for duplicates.
- I understand that dispatcherd is open source software provided for free and that I might not receive a timely response.
Feature type
New Feature
Feature Summary
We loop over workers in a very linear way:
dispatcherd/dispatcherd/service/queuer.py
Lines 22 to 24 in 40e5233
| for candidate_worker in self.workers: | |
| if (not candidate_worker.current_task) and candidate_worker.is_ready: | |
| return candidate_worker |
It might be confusing that this isn't obviously a list... but in practice it is. We just go from the first element to last.
This isn't ideal, because it means that the first worker gets all the tasks and the last worker doesn't get many at all. This is bad for connection management and things like that.
In AWX, we do random.shuffle on the workers before looping over them. So that's 1 option. To summarize multiple options for finding a free worker:
- The dumb linear approach or going in-order through all workers
- Shuffling workers
- Sorting workers in order of longest time idle
The option (3) seems to be the ideal to me, because it works best with the user's database connection policy.
Steps to reproduce
n/a
Current results
No response
Sugested feature result
n/a
Additional information
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request