Queuing behavior is opaque and unconfigurable.

[dreid]

The behavior of ampoule.pool.ProcessPool when all the workers are busy is to add the requests to a list of queued requests and to pull work from there as workers become free.

There are a few cons to this current approach:

• There is no public interface for seeing how many jobs are queued (the queue itself is not public unlike the processes, busy, ready` lists.)
• There is no public interface for seeing which jobs are in queued.
• There is no way to configure a maximum size of the queue to allow for load shedding.
• There is no way to use a priority queue to prioritize certain jobs.
• There is no way to know how long a dispatched job was queued before it was dispatched to a worker.

These problems could mostly be addressed by a bring your own queue strategy (byoq) instead of simply using a list and the heapq module.

Pool could instead take an object with a queue.Queue like interface that could be extended via subclassing or composition. Allowing implementations to be instrumented to improve telemetry of process pool, or to implement behaviors of like prioritization or load shedding.