feat(task queues): created initial draft for priority and fairness #4080
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📖 Docs PR preview links
|
flippedcoder
force-pushed
the
mm/task-queue-priority-fairness
branch
from
27a91b6 to
a62958a
Compare
flippedcoder
force-pushed
the
mm/task-queue-priority-fairness
branch
from
d412c87 to
db511e7
Compare
| [Task Queue Priority](#task-queue-priority) and [Task Queue Fairness](#task-queue-fairness) are two ways to manage the distribution of work within Task Queues. Priority focuses on how Tasks are prioritized within a Task Queue and Fairness aims to prevent one set of prioritized Tasks from blocking others within the same Task Queue. | ||
|
|
||
| :::tip Support, stability, and dependency info | ||
| Priority and Fairness is currently in [pre-release](/evaluate/development-production-features/release-stages#pre-release). |
There was a problem hiding this comment.
I think these docs will go live when we do the public preview release. Will confirm with @bchav .
|
|
||
| If you're using Temporal Cloud, contact Temporal Support or your Temporal account team to enable this feature for your cloud Namespaces. | ||
|
|
||
| If you're self-hosting Temporal, use the latest pre-release development server and set `matching.useNewMatcher` to `true` in the [dynamic config](https://github.com/temporalio/temporal/blob/a3a53266c002ae33b630a41977274f8b5b587031/common/dynamicconfig/constants.go#L1345-L1348) on the relevant Task Queues or Namespaces. |
There was a problem hiding this comment.
@bchav Just checking if this is accurate or not.
|
|
||
| When you use Worker Versioning and you're moving Workflows from one version to another, the ordering of Workflow Tasks that are moved to the next version is undefined. Tasks redirected to a new Worker version may not be treated fairly with respect to each other or Tasks that aren't redirected. | ||
|
|
||
| There isn't a limit on the number of fairness keys you can use, but their accuracy can degrade as you add more. |
There was a problem hiding this comment.
@bchav Do we want to include a number here? Like the accuracy doesn't degrade until ~100,000 or something?
lennessyy
left a comment
lennessyy
left a comment
There was a problem hiding this comment.
In addition to this page, I think these options should also be called out in the individual SDKs. We can just link to this page, but if they are not called out there then developers who only look at their language-specific resources will not find this.
|
|
||
| ## Task Queue priority | ||
|
|
||
| **Task Queue priority** lets you control the execution order of Workflows, Activities, and Child Workflows based on assigned priority values within a *single* Task Queue. Each priority level acts as a "virtual" task queue that prioritizes Tasks within a single Task Queue. |
There was a problem hiding this comment.
| **Task Queue priority** lets you control the execution order of Workflows, Activities, and Child Workflows based on assigned priority values within a *single* Task Queue. Each priority level acts as a "virtual" task queue that prioritizes Tasks within a single Task Queue. | |
| **Task Queue Priority** lets you control the execution order of Workflows, Activities, and Child Workflows based on assigned priority values within a *single* Task Queue. Each priority level acts as a "virtual" task queue that prioritizes Tasks within a single Task Queue. |
Is Priority a new primitive and should be initial capitalized? The first paragraph of the page seems to treat it this way
|
|
||
| ### When to use priority | ||
|
|
||
| If you need a way to specify the order your Tasks execute in, you can use priority to manage that. Priority lets you differentiate between your Tasks, like batch and real-time for example, so that you can allocate resources more effectively. You can also use this as a way to run urgent Tasks immediately and override others. |
There was a problem hiding this comment.
like batch and real-time for example
I am not sure I understand what this means. Like giving all the tasks in the same batch the same priority to batch them? Or having a high priority for real-time tasks?
I think it would be nice to give a more concrete example for each use case. Maybe we can say something like "For example, if you are running an E-commerce platform, you may want to prioritize payment related workflows over less time-sensitive workflows like internal inventory management." This could be for "use this as a way to run urgent Tasks immediately and override others."
|
|
||
| ### How to use priority | ||
|
|
||
| You can select a priority level by setting the _priority key_ to a value within the integer range `[1,5]`. A lower value implies higher priority, so `1` is the highest priority level. The default priority, if unspecified, is `3`. |
There was a problem hiding this comment.
| You can select a priority level by setting the _priority key_ to a value within the integer range `[1,5]`. A lower value implies higher priority, so `1` is the highest priority level. The default priority, if unspecified, is `3`. | |
| You can select a priority level by setting the _priority key_ to a value within the integer range `[1,5]`. A lower value implies higher priority, so `1` is the highest priority level. If you don't specify a Priority, a Task defaults to a Priority of`3`. |
To avoid the confusion that that is a "default" default priority that you need to set.
|
|
||
| When you set a priority level within your Task Queues, this means that they will __all__ be processed in priority order. For example, all of your priority level `1` Tasks will execute before your priority level `2` Tasks and so on. So your lower priority Tasks will be blocked until the higher priority Tasks finish running. Tasks with the same priority level are scheduled to run in first-in-first-out (FIFO) order. If you need more flexibility to allocate resources to lower priority Tasks sooner, check out [the fairness section](#task-queue-fairness). | ||
|
|
||
| Activities will inherit the priority level of their Workflow if a separate Activity priority level isn't set. |
There was a problem hiding this comment.
I think this is worth mentioning earlier when we talked about default priority
| When you update priority levels, the Task Queues will only reflect these changes for Tasks that haven't dispatched yet, not Tasks that are already queued. | ||
|
|
||
| :::note | ||
| [Sticky tasks](/sticky-execution) ignore priority settings. |
There was a problem hiding this comment.
How does it ignore it? As in, if there are sticky tasks that are in a sticky queue, they get executed ahead of higher priority tasks? Even if the initial task is low priority? I think this could use a little more detail
| - Multi-tenant applications with big and small tenants where small tenants shouldn't be blocked by big ones. That means any single user should be able to use all available resources rather than being rate limited at a fixed level. | ||
| - Assigning Tasks to different capacity bands and then, for example, dispatching 80% from one band and 20% from another without limiting overall capacity when one band is empty. | ||
|
|
||
| Fairness is only avaliable as a paid feature in Temporal Cloud, so contact Temporal Support or your Temporal account team to enable this feature for your cloud Namespaces. Fairness can be enabled or disabled at the Namespace level and billing will be disabled at the next calendar hour after it is disabled. |
There was a problem hiding this comment.
Does this mean this is not available to self-hosting customers? I feel like we'd need to be explicit if that is the case
|
|
||
| ### When to use fairness | ||
|
|
||
| Fairness applies to backlogged Tasks when there isn't sufficient Worker capacity to dispatch Tasks immediately. This is applicable if you want to distribute a finite resource across all of your Tasks without blockers. If all Tasks can be dispatched immediately, then you don't need to use fairness. |
There was a problem hiding this comment.
I think we should either explain the concepts of backlog again here, or link to the the related concept in the Task Queue page. This explanation here kind of makes it sound like backlogs are separate from Task Queues.
|
|
||
| Fairness attempts to distribute all the Tasks from a higher priority level first before distributing any from a lower priority. The fairness weights allow for unequal distribution among tenants or for allocating fractions of capacity to different levels or types of Tasks. | ||
|
|
||
| Fairness applies within each priority level. It sequences Tasks in the Task Queue probabilistically using a weighted round-robin, based on: |
There was a problem hiding this comment.
I only know round-robin from tennis tournaments 😅 what does it mean here?
|
|
||
| ### How fairness works | ||
|
|
||
| Fairness sequences Tasks at each priority level based on _fairness keys_ and their _fairness weight_ within a Task Queue. The fairness keys are used to describe your Task structure. A fairness key can correspond to unique applications, tenants, or other groupings useful to your workloads. The Tasks associated with each fairness key are dispatched based on the _fairness weight_ that has been assigned to the key. Using the fairness keys and their corresponding fairness weights lets you define tiers with weighted capacities, like high, medium, and low, so that higher priority tasks don't overwhelm your resources and block lower priority tasks. |
There was a problem hiding this comment.
I think the fairness section is still a bit muddling to me. I read this over twice, but i still don't know when and how fairness
I think we need more concrete examples here. Like say I have a task queue with tasks of priority 1 and priority 3. Most of the backlog is priority 3 because of the set priority. If I set a fairness weight here (where do I set the weight?), how does that help tasks with priority 3 get unblocked?
What does this PR do?
This adds a new page to explain Task Queue priority and fairness.