Skip to content

Worker rotation options #156

@AlanCoding

Description

@AlanCoding

Please confirm the following

  • I agree to follow this project's code of conduct.
  • I have checked the current issues for duplicates.
  • I understand that dispatcherd is open source software provided for free and that I might not receive a timely response.

Feature type

New Feature

Feature Summary

We loop over workers in a very linear way:

for candidate_worker in self.workers:
if (not candidate_worker.current_task) and candidate_worker.is_ready:
return candidate_worker

It might be confusing that this isn't obviously a list... but in practice it is. We just go from the first element to last.

This isn't ideal, because it means that the first worker gets all the tasks and the last worker doesn't get many at all. This is bad for connection management and things like that.

In AWX, we do random.shuffle on the workers before looping over them. So that's 1 option. To summarize multiple options for finding a free worker:

  1. The dumb linear approach or going in-order through all workers
  2. Shuffling workers
  3. Sorting workers in order of longest time idle

The option (3) seems to be the ideal to me, because it works best with the user's database connection policy.

Steps to reproduce

n/a

Current results

No response

Sugested feature result

n/a

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions