feat: adds the preemptible task hint

CHANGELOG.md

@@ -17,6 +17,11 @@ Keep the changelog pleasant to read in the text editor:
 + Properly indent blocks.
 -->
 
+version 1.4.0
+---------------------------
+
++ Introduced the `preemptible` task hint and `preemptible_only` task requirement ([#728](https://github.com/openwdl/wdl/pull/728)).
+
 version 1.3.0
 ---------------------------
 

README.md

@@ -8,8 +8,8 @@
 |-----------|------------------|
 | MiniWDL   | [![MiniWDL Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/openwdl/wdl/wdl-1.4/shields/miniwdl.json)](https://github.com/openwdl/wdl/actions/runs/29465188571) |
 | Sprocket   | [![Sprocket Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/openwdl/wdl/wdl-1.4/shields/sprocket.json)](https://github.com/openwdl/wdl/actions/runs/29465188597) |
-| Toil      | [![Toil Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/openwdl/wdl/wdl-1.4/shields/toil.json)](https://github.com/openwdl/wdl/actions/runs/29465188570) |
-| Cromwell  | [![Cromwell Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/openwdl/wdl/wdl-1.4/shields/cromwell.json)](https://github.com/openwdl/wdl/actions/runs/28450565217) |
+| Toil      | [![Toil Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/openwdl/wdl/preemptible-hint/shields/toil.json)](https://github.com/openwdl/wdl/actions/runs/29525409350) |
+| Cromwell  | [![Cromwell Tests](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/openwdl/wdl/preemptible-hint/shields/cromwell.json)](https://github.com/openwdl/wdl/actions/runs/29465517910) |
 
 
 The **Workflow Description Language (WDL)** (pronounced as _/hwɪdl/_ or "whittle" with a 'd') is an open standard for describing data processing workflows using a human-readable/writeable syntax.

SPEC.md

@@ -112,6 +112,7 @@ Revisions to this specification are made periodically in order to correct errors
         - [Hardware Accelerators (`gpu` and `fpga`)](#hardware-accelerators-gpu-and--fpga)
         - [`disks`](#disks)
         - [`max_retries`](#max_retries)
+        - [✨ `preemptible_only`](#-preemptible_only)
         - [`return_codes`](#return_codes)
     - [Hints Section](#-hints-section)
       - [Hints-scoped types](#hints-scoped-types)
@@ -124,6 +125,7 @@ Revisions to this specification are made periodically in order to correct errors
         - [`localization_optional`](#localization_optional)
         - [`inputs`](#inputs)
         - [`outputs`](#outputs)
+        - [✨ `preemptible`](#-preemptible)
       - [Compute Environments](#compute-environments)
       - [Conventions and Best Practices](#conventions-and-best-practices)
     - [🗑 Runtime Section](#-runtime-section)
@@ -5785,11 +5787,13 @@ Test config:
 * Default value: `0`
 * Alias: `maxRetries`
 
-The `max_retries` attribute specifies the maximum number of times a task should be retried in the event of failure. The execution engine must retry the task at least once and up to (but not exceeding) the specified number of attempts.
+The `max_retries` attribute specifies the maximum number of times a task should be retried following a failure other than preemption. The execution engine must retry the task at least once and up to (but not exceeding) the specified number of attempts.
+
+A retry following preemption does not count against `max_retries`. If the engine honors the [`preemptible`](#-preemptible) hint, that hint governs the number of preemptible attempts. Starting a non-preemptible attempt after exhausting those attempts does count as one retry.
 
 The execution engine may choose to define an upper bound (>= 1) on the number of retry attempts that it permits.
 
-A value of `0` means that the task as not retryable, and therefore any failure in the task should never result in a retry by the execution engine, and the final status of the task should remain the same.
+A value of `0` means that a failure other than preemption must not result in a retry. It does not prevent retries on preemptible instances following preemption, but it prevents the engine from starting a non-preemptible attempt after the `preemptible` attempts are exhausted.
 
 ```wdl
 task max_retries_test {
@@ -5800,6 +5804,32 @@ task max_retries_test {
 }
 ```
 
+##### ✨ `preemptible_only`
+
+* Accepted type: `Boolean`
+* Default value: `false`
+
+A preemptible instance is a compute instance that the execution environment may reclaim before the task completes. Reclamation of an instance in this way is a *preemption*, which is distinct from a failure of the task's command.
+
+When `preemptible_only` is `false`, this requirement has no effect. When `preemptible_only` is `true`, the execution engine must run every attempt on a preemptible instance. If the engine cannot provide a preemptible instance, the task must fail before the command is instantiated.
+
+This requirement does not require the engine to honor the [`preemptible`](#-preemptible) hint. If the engine honors that hint and exhausts the suggested number of preemptible attempts, the task fails because this requirement prohibits a non-preemptible attempt. If the hint is absent or ignored, the engine determines whether to retry after preemption. A retry following preemption does not count against [`max_retries`](#max_retries), while a retry following any other failure does.
+
+```wdl
+version 1.4
+
+task preemptible_only_example {
+  command <<< echo "hello" >>>
+
+  requirements {
+    preemptible_only: true
+    max_retries: 2
+  }
+}
+```
+
+Every attempt must run on a preemptible instance. The engine determines whether to retry after preemption, and such retries do not count against `max_retries`. Independently, up to two attempts that fail for reasons other than preemption may be retried on preemptible instances.
+
 ##### `return_codes`
 
 * Accepted types:
@@ -6152,6 +6182,39 @@ Reserved input-specific attributes:
 
 Provides output-specific hints. Each key must refer to a parameter defined in the task's [`output`](#task-outputs) section. A key may also use dotted notation to refer to a specific member of a struct output.
 
+##### ✨ `preemptible`
+
+* Accepted type: `Int`
+* Default value: `0`
+
+Suggests the maximum number of attempts that may use a [preemptible instance](#-preemptible_only). An execution engine may ignore this hint. When [`preemptible_only`](#-preemptible_only) is `false`, an engine that ignores the hint may run the initial attempt on a non-preemptible instance. This does not consume a retry permitted by [`max_retries`](#max_retries).
+
+When an engine honors this hint, an attempt that ends in preemption counts against `preemptible` and does not count against `max_retries`. An attempt that ends in any other failure counts against `max_retries` and does not count against `preemptible`, regardless of the instance type on which it ran. The two limits are independent.
+
+When the engine honors this hint and `preemptible_only` is `false`, the engine must not start another preemptible attempt once `preemptible` attempts have ended in preemption. If at least one retry remains under `max_retries`, the engine must use one retry to continue on a non-preemptible instance. If no retry remains, the task fails. In particular, `max_retries: 0` never permits a non-preemptible attempt after an attempt ends in preemption, but an engine may still ignore the hint and run the initial attempt on a non-preemptible instance.
+
+A value of `0` suggests that the task should not run on a preemptible instance.
+
+When [`preemptible_only`](#-preemptible_only) is `true`, the engine must run every attempt on a preemptible instance even if this hint is absent, ignored, or has a value of `0`. If the engine honors a positive value for this hint and exhausts the suggested number of attempts, the task fails because `preemptible_only` prohibits a non-preemptible attempt.
+
+```wdl
+version 1.4
+
+task preemptible_example {
+  command <<< echo "hello" >>>
+
+  requirements {
+    max_retries: 2
+  }
+
+  hints {
+    preemptible: 3
+  }
+}
+```
+
+An engine may ignore the hint and run the initial attempt on a non-preemptible instance. An engine that honors the hint may run up to three attempts on a preemptible instance. If all three are preempted, none of those preemptions count against `max_retries`. The engine then uses one of the two retries to continue on a non-preemptible instance. One retry remains if that attempt fails for a reason other than preemption.
+
 #### Compute Environments
 
 The `hints` section should be used to provide hints that are specific to different compute environments such as HPC systems or cloud platforms. Attributes for a compute environment should be specified in a `hints` value, in which any of the [reserved hints](#reserved-task-hints) are allowed to override the values specified at the task level (if any), and other attributes are platform-specific.
@@ -6349,6 +6412,8 @@ This information is provided by the `task` variable, which is implicitly defined
 * `fpga`: An `Array[String]` with one specification per allocated FPGA. The specification is execution engine-specific. If no FPGAs were allocated, then the value must be an empty array.
 * `disks`: A `Map[String, Int]` with one entry for each disk mount point. The key is the mount point and the value is the initial amount of disk space allocated, in bytes. The execution engine must, at a minimum, provide one entry for each disk mount point requested, but may provide more. The amount of disk space available for a given mount point may increase during the lifetime of the task (e.g., autoscaling volumes provided by some cloud services).
 * `max_retries`: An `Int` with the maximum number of retry attempts.
+* `preemptible` ✨: An `Int` with the maximum number of attempts that may use a preemptible instance.
+* `preemptible_only` ✨: A `Boolean` indicating whether the task may run only on preemptible instances.
 * `attempt`: An `Int` with the current task attempt. The value must be `0` the first time the task is executed, and incremented by `1` each time the task is retried (if any).
 * `previous`: A [hidden type](#taskprevious-hidden-scoped-type) containing the computed requirements from the previous task attempt. All fields are `None` on the first try.
 * `end_time`: An `Int?` whose value is the time by which the task must be completed, as a [Unix time stamp](https://en.wikipedia.org/wiki/Unix_time). A value of `0` means that the execution engine does not impose a time limit. A value of `None` means that the execution engine cannot determine whether the runtime of the task is limited. A positive value is a guarantee that the task will be preempted at the specified time, but is *not* a guarantee that the task won't be preempted earlier.

shields/cromwell.json

@@ -1,6 +1,6 @@
 {
   "schemaVersion": 1,
   "label": "Cromwell / WDL 1.4",
-  "message": "63/176 passed",
+  "message": "59/172 passed",
   "color": "red"
 }

shields/toil.json

@@ -1,6 +1,6 @@
 {
   "schemaVersion": 1,
   "label": "Toil / WDL 1.4",
-  "message": "137/172 passed",
+  "message": "136/172 passed",
   "color": "yellow"
 }