Fairness (#617)

**What changed?**

Already [discussed and approved previously
here](#513).

**Breaking changes**

N/A

**Server PR**

https://github.com/temporalio/temporal/tree/feature/fairness

---------

Co-authored-by: David Reiss <[email protected]>

Author: stephanos

openapi/openapiv2.json

@@ -13127,10 +13127,19 @@
         "priorityKey": {
           "type": "integer",
           "format": "int32",
-          "description": "Priority key is a positive integer from 1 to n, where smaller integers\ncorrespond to higher priorities (tasks run sooner). In general, tasks in\na queue should be processed in close to priority order, although small\ndeviations are possible.\n\nThe maximum priority value (minimum priority) is determined by server\nconfiguration, and defaults to 5.\n\nThe default priority is (min+max)/2. With the default max of 5 and min of\n1, that comes out to 3."
+          "description": "Priority key is a positive integer from 1 to n, where smaller integers\ncorrespond to higher priorities (tasks run sooner). In general, tasks in\na queue should be processed in close to priority order, although small\ndeviations are possible.\n\nThe maximum priority value (minimum priority) is determined by server\nconfiguration, and defaults to 5.\n\nIf priority is not present (or zero), then the effective priority will be\nthe default priority, which is is calculated by (min+max)/2. With the\ndefault max of 5, and min of 1, that comes out to 3."
+        },
+        "fairnessKey": {
+          "type": "string",
+          "description": "Fairness key is a short string that's used as a key for a fairness\nbalancing mechanism. It may correspond to a tenant id, or to a fixed\nstring like \"high\" or \"low\". The default is the empty string.\n\nThe fairness mechanism attempts to dispatch tasks for a given key in\nproportion to its weight. For example, using a thousand distinct tenant\nids, each with a weight of 1.0 (the default) will result in each tenant\ngetting a roughly equal share of task dispatch throughput.\n\n(Note: this does not imply equal share of worker capacity! Fairness\ndecisions are made based on queue statistics, not\ncurrent worker load.)\n\nAs another example, using keys \"high\" and \"low\" with weight 9.0 and 1.0\nrespectively will prefer dispatching \"high\" tasks over \"low\" tasks at a\n9:1 ratio, while allowing either key to use all worker capacity if the\nother is not present.\n\nAll fairness mechanisms, including rate limits, are best-effort and\nprobabilistic. The results may not match what a \"perfect\" algorithm with\ninfinite resources would produce. The more unique keys are used, the less\naccurate the results will be.\n\nFairness keys are limited to 64 bytes."
+        },
+        "fairnessWeight": {
+          "type": "number",
+          "format": "float",
+          "description": "Fairness weight for a task can come from multiple sources for\nflexibility. From highest to lowest precedence:\n1. Weights for a small set of keys can be overridden in task queue\n   configuration with an API.\n2. It can be attached to the workflow/activity in this field.\n3. The default weight of 1.0 will be used.\n\nWeight values are clamped to the range [0.001, 1000]."
         }
       },
-      "description": "Priority contains metadata that controls relative ordering of task processing\nwhen tasks are backlogged in a queue. Initially, Priority will be used in\nactivity and workflow task queues, which are typically where backlogs exist.\nOther queues in the server (such as transfer and timer queues) and rate\nlimiting decisions do not use Priority, but may in the future.\n\nPriority is attached to workflows and activities. Activities and child\nworkflows inherit Priority from the workflow that created them, but may\noverride fields when they are started or modified. For each field of a\nPriority on an activity/workflow, not present or equal to zero/empty string\nmeans to inherit the value from the calling workflow, or if there is no\ncalling workflow, then use the default (documented below).\n\nDespite being named \"Priority\", this message will also contains fields that\ncontrol \"fairness\" mechanisms.\n\nThe overall semantics of Priority are:\n1. First, consider \"priority_key\": lower number goes first.\n(more will be added here later)"
+      "description": "Priority contains metadata that controls relative ordering of task processing\nwhen tasks are backed up in a queue. Initially, Priority will be used in\nmatching (workflow and activity) task queues. Later it may be used in history\ntask queues and in rate limiting decisions.\n\nPriority is attached to workflows and activities. By default, activities\ninherit Priority from the workflow that created them, but may override fields\nwhen an activity is started or modified.\n\nDespite being named \"Priority\", this message also contains fields that\ncontrol \"fairness\" mechanisms.\n\nFor all fields, the field not present or equal to zero/empty string means to\ninherit the value from the calling workflow, or if there is no calling\nworkflow, then use the default value.\n\nFor all fields other than fairness_key, the zero value isn't meaningful so\nthere's no confusion between inherit/default and a meaningful value. For\nfairness_key, the empty string will be interpreted as \"inherit\". This means\nthat if a workflow has a non-empty fairness key, you can't override the\nfairness key of its activity to the empty string.\n\nThe overall semantics of Priority are:\n1. First, consider \"priority\": higher priority (lower number) goes first.\n2. Then, consider fairness: try to dispatch tasks for different fairness keys\n   in proportion to their weight.\n\nApplications may use any subset of mechanisms that are useful to them and\nleave the other fields to use default values.\n\nNot all queues in the system may support the \"full\" semantics of all priority\nfields. (Currently only support in matching task queues is planned.)"
     },
     "v1ProtocolMessageCommandAttributes": {
       "type": "object",

openapi/openapiv3.yaml

@@ -9909,29 +9909,82 @@ components:
              The maximum priority value (minimum priority) is determined by server
              configuration, and defaults to 5.
 
-             The default priority is (min+max)/2. With the default max of 5 and min of
-             1, that comes out to 3.
+             If priority is not present (or zero), then the effective priority will be
+             the default priority, which is is calculated by (min+max)/2. With the
+             default max of 5, and min of 1, that comes out to 3.
           format: int32
+        fairnessKey:
+          type: string
+          description: |-
+            Fairness key is a short string that's used as a key for a fairness
+             balancing mechanism. It may correspond to a tenant id, or to a fixed
+             string like "high" or "low". The default is the empty string.
+
+             The fairness mechanism attempts to dispatch tasks for a given key in
+             proportion to its weight. For example, using a thousand distinct tenant
+             ids, each with a weight of 1.0 (the default) will result in each tenant
+             getting a roughly equal share of task dispatch throughput.
+
+             (Note: this does not imply equal share of worker capacity! Fairness
+             decisions are made based on queue statistics, not
+             current worker load.)
+
+             As another example, using keys "high" and "low" with weight 9.0 and 1.0
+             respectively will prefer dispatching "high" tasks over "low" tasks at a
+             9:1 ratio, while allowing either key to use all worker capacity if the
+             other is not present.
+
+             All fairness mechanisms, including rate limits, are best-effort and
+             probabilistic. The results may not match what a "perfect" algorithm with
+             infinite resources would produce. The more unique keys are used, the less
+             accurate the results will be.
+
+             Fairness keys are limited to 64 bytes.
+        fairnessWeight:
+          type: number
+          description: |-
+            Fairness weight for a task can come from multiple sources for
+             flexibility. From highest to lowest precedence:
+             1. Weights for a small set of keys can be overridden in task queue
+                configuration with an API.
+             2. It can be attached to the workflow/activity in this field.
+             3. The default weight of 1.0 will be used.
+
+             Weight values are clamped to the range [0.001, 1000].
+          format: float
       description: |-
         Priority contains metadata that controls relative ordering of task processing
-         when tasks are backlogged in a queue. Initially, Priority will be used in
-         activity and workflow task queues, which are typically where backlogs exist.
-         Other queues in the server (such as transfer and timer queues) and rate
-         limiting decisions do not use Priority, but may in the future.
-
-         Priority is attached to workflows and activities. Activities and child
-         workflows inherit Priority from the workflow that created them, but may
-         override fields when they are started or modified. For each field of a
-         Priority on an activity/workflow, not present or equal to zero/empty string
-         means to inherit the value from the calling workflow, or if there is no
-         calling workflow, then use the default (documented below).
-
-         Despite being named "Priority", this message will also contains fields that
+         when tasks are backed up in a queue. Initially, Priority will be used in
+         matching (workflow and activity) task queues. Later it may be used in history
+         task queues and in rate limiting decisions.
+
+         Priority is attached to workflows and activities. By default, activities
+         inherit Priority from the workflow that created them, but may override fields
+         when an activity is started or modified.
+
+         Despite being named "Priority", this message also contains fields that
          control "fairness" mechanisms.
 
+         For all fields, the field not present or equal to zero/empty string means to
+         inherit the value from the calling workflow, or if there is no calling
+         workflow, then use the default value.
+
+         For all fields other than fairness_key, the zero value isn't meaningful so
+         there's no confusion between inherit/default and a meaningful value. For
+         fairness_key, the empty string will be interpreted as "inherit". This means
+         that if a workflow has a non-empty fairness key, you can't override the
+         fairness key of its activity to the empty string.
+
          The overall semantics of Priority are:
-         1. First, consider "priority_key": lower number goes first.
-         (more will be added here later)
+         1. First, consider "priority": higher priority (lower number) goes first.
+         2. Then, consider fairness: try to dispatch tasks for different fairness keys
+            in proportion to their weight.
+
+         Applications may use any subset of mechanisms that are useful to them and
+         leave the other fields to use default values.
+
+         Not all queues in the system may support the "full" semantics of all priority
+         fields. (Currently only support in matching task queues is planned.)
     QueryRejected:
       type: object
       properties:

temporal/api/common/v1/message.proto

@@ -239,24 +239,37 @@ message Link {
 }
 
 // Priority contains metadata that controls relative ordering of task processing
-// when tasks are backlogged in a queue. Initially, Priority will be used in
-// activity and workflow task queues, which are typically where backlogs exist.
-// Other queues in the server (such as transfer and timer queues) and rate
-// limiting decisions do not use Priority, but may in the future.
+// when tasks are backed up in a queue. Initially, Priority will be used in
+// matching (workflow and activity) task queues. Later it may be used in history
+// task queues and in rate limiting decisions.
 //
-// Priority is attached to workflows and activities. Activities and child
-// workflows inherit Priority from the workflow that created them, but may
-// override fields when they are started or modified. For each field of a
-// Priority on an activity/workflow, not present or equal to zero/empty string
-// means to inherit the value from the calling workflow, or if there is no
-// calling workflow, then use the default (documented below).
+// Priority is attached to workflows and activities. By default, activities
+// inherit Priority from the workflow that created them, but may override fields
+// when an activity is started or modified.
 //
-// Despite being named "Priority", this message will also contains fields that
+// Despite being named "Priority", this message also contains fields that
 // control "fairness" mechanisms.
 //
+// For all fields, the field not present or equal to zero/empty string means to
+// inherit the value from the calling workflow, or if there is no calling
+// workflow, then use the default value.
+//
+// For all fields other than fairness_key, the zero value isn't meaningful so
+// there's no confusion between inherit/default and a meaningful value. For
+// fairness_key, the empty string will be interpreted as "inherit". This means
+// that if a workflow has a non-empty fairness key, you can't override the
+// fairness key of its activity to the empty string.
+//
 // The overall semantics of Priority are:
-// 1. First, consider "priority_key": lower number goes first.
-// (more will be added here later)
+// 1. First, consider "priority": higher priority (lower number) goes first.
+// 2. Then, consider fairness: try to dispatch tasks for different fairness keys
+//    in proportion to their weight.
+//
+// Applications may use any subset of mechanisms that are useful to them and
+// leave the other fields to use default values.
+//
+// Not all queues in the system may support the "full" semantics of all priority
+// fields. (Currently only support in matching task queues is planned.)
 message Priority {
     // Priority key is a positive integer from 1 to n, where smaller integers
     // correspond to higher priorities (tasks run sooner). In general, tasks in
@@ -266,9 +279,46 @@ message Priority {
     // The maximum priority value (minimum priority) is determined by server
     // configuration, and defaults to 5.
     //
-    // The default priority is (min+max)/2. With the default max of 5 and min of
-    // 1, that comes out to 3.
+    // If priority is not present (or zero), then the effective priority will be
+    // the default priority, which is is calculated by (min+max)/2. With the
+    // default max of 5, and min of 1, that comes out to 3.
     int32 priority_key = 1;
+
+    // Fairness key is a short string that's used as a key for a fairness
+    // balancing mechanism. It may correspond to a tenant id, or to a fixed
+    // string like "high" or "low". The default is the empty string.
+    //
+    // The fairness mechanism attempts to dispatch tasks for a given key in
+    // proportion to its weight. For example, using a thousand distinct tenant
+    // ids, each with a weight of 1.0 (the default) will result in each tenant
+    // getting a roughly equal share of task dispatch throughput.
+    //
+    // (Note: this does not imply equal share of worker capacity! Fairness
+    // decisions are made based on queue statistics, not
+    // current worker load.)
+    //
+    // As another example, using keys "high" and "low" with weight 9.0 and 1.0
+    // respectively will prefer dispatching "high" tasks over "low" tasks at a
+    // 9:1 ratio, while allowing either key to use all worker capacity if the
+    // other is not present.
+    //
+    // All fairness mechanisms, including rate limits, are best-effort and
+    // probabilistic. The results may not match what a "perfect" algorithm with
+    // infinite resources would produce. The more unique keys are used, the less
+    // accurate the results will be.
+    //
+    // Fairness keys are limited to 64 bytes.
+    string fairness_key = 2;
+
+    // Fairness weight for a task can come from multiple sources for
+    // flexibility. From highest to lowest precedence:
+    // 1. Weights for a small set of keys can be overridden in task queue
+    //    configuration with an API.
+    // 2. It can be attached to the workflow/activity in this field.
+    // 3. The default weight of 1.0 will be used.
+    //
+    // Weight values are clamped to the range [0.001, 1000].
+    float fairness_weight = 3;
 }
 
 // This is used to send commands to a specific worker or a group of workers.