Workflow error handling updates (#175) (#176)

* Workflow error handling updates

Signed-off-by: Tihomir Surdilovic <[email protected]>

* Updates to error and retry definitions

Signed-off-by: Tihomir Surdilovic <[email protected]>

* Update to error handling

Signed-off-by: Tihomir Surdilovic <[email protected]>

* small typo fix

Signed-off-by: Tihomir Surdilovic <[email protected]>

* another small typo fix

Signed-off-by: Tihomir Surdilovic <[email protected]>

* yet another typo fix

Signed-off-by: Tihomir Surdilovic <[email protected]>

* Update after review comments

Signed-off-by: Tihomir Surdilovic <[email protected]>

* Update to roadmap document

Signed-off-by: Tihomir Surdilovic <[email protected]>

Author: Tihomir Surdilovic

examples/examples-argo.md

@@ -696,6 +696,11 @@ functions:
   metadata:
     image: python:alpine3.6
     command: python
+retries:
+- name: All workflow errors retry strategy
+  maxAttempts: 10
+  multiplier: PT2M
+  delay: PT1M
 states:
 - name: retry-backoff
   type: operation
@@ -707,11 +712,11 @@ states:
       parameters:
         args:
         - import random; import sys; exit_code = random.choice([0, 1, 1]); sys.exit(exit_code)
-  retry:
-  - expression: "{{ $.[?(@.exitcode == '1')] }}"
-    maxAttempts: 10
-    multiplier: PT2M
-    interval: PT1M
+  onErrors:
+  - error: "*"
+    retryRef: All workflow errors retry strategy
+    end:
+      kind: default
   end:
     kind: default
 ```
@@ -829,10 +834,11 @@ states:
 [Argo Example](https://github.com/argoproj/argo/tree/master/examples#exit-handlers)
 
 *Note*: With Serverless Workflow specification we can handle Argos "onExit" functionality
-in couple of ways. One is the "onError" functionality to catch errors and transition to the 
-"error" part of the workflow. Another is to send an event at the end of workflow execution 
+in couple of ways. One is the "onErrors" functionality to define errors and transition to the parts
+of workflow which is capable of handling the errors. 
+Another is to send an event at the end of workflow execution 
 which includes the workflow status. This event can then trigger execution other workflows
-that can handle each status. For this example we use the "onError" definition.
+that can handle each status. For this example we use the "onErrors" definition.
 
 <table>
 <tr>
@@ -919,12 +925,10 @@ states:
       refName: intentional-fail-function
       parameters:
         args: echo intentional failure; exit 1
-  onError:
-  - expression: "{{ $.errors[0] }}"
-    errorDataFilter:
-      dataOutputPath: "{{ $.exit-code }}"
-  transition:
-    nextState: send-email-state
+  onErrors:
+  - error: "*"
+    transition:
+      nextState: send-email-state
 - name: send-email-state
   type: operation
   actions:

examples/examples-brigade.md

@@ -180,10 +180,8 @@ states:
         refName: consoleLogFunction
         parameters:
           log: done
-  onError:
-  - expression:
-      language: spel
-      body: "{{ $.exceptions[0] }}"
+  onErrors:
+  - error: "*"
     transition:
       nextState: HandleErrorState
   end:

examples/examples.md

@@ -891,10 +891,9 @@ states:
 #### Description
 
 In this example we show off the states error handling capability. The workflow data input that's passed in contains
-missing order information that causes the function in the "ProvisionOrder" state to throw a runtime exception. With the "onError" expression we
-can transition the workflow to different error handling states depending on the error thrown. Each type of error
-in this example is handled by simple delay states, each including an error data filter which sets the exception info as their
-data output. If no error is caught the workflow can transition to the "ApplyOrder" state.
+missing order information that causes the function in the "ProvisionOrder" state to throw a runtime exception. With the "onErrors" definition we
+can transition the workflow to different error handling states. Each type of error
+in this example is handled by simple delay states. If no errors are encountered the workflow can transition to the "ApplyOrder" state.
 
 Workflow data is assumed to me:
 
@@ -956,32 +955,32 @@ The data output of the workflow contains the information of the exception caught
           }
        }
     ],
-    "onError": [
+    "stateDataFilter": {
+       "dataOutputPath": "{{ $.exceptions }}"
+    },
+    "transition": {
+       "nextState":"ApplyOrder"
+    },
+    "onErrors": [
        {
-         "expression": "{{ $.exceptions[?(@.name == 'MissingOrderIdException')] }}",
+         "error": "Missing order id",
          "transition": {
            "nextState": "MissingId"
          }
        },
        {
-         "expression": "{{ $.exceptions[?(@.name == 'MissingOrderItemException')] }}",
+         "error": "Missing order item",
          "transition": {
            "nextState": "MissingItem"
          }
        },
        {
-        "expression": "{{ $.exceptions[?(@.name == 'MissingOrderQuantityException')] }}",
+        "error": "Missing order quantity",
         "transition": {
           "nextState": "MissingQuantity"
         }
        }
-    ],
-    "stateDataFilter": {
-       "dataOutputPath": "{{ $.exceptions }}"
-    },
-    "transition": {
-       "nextState":"ApplyOrder"
-    }
+    ]
 },
 {
    "name": "MissingId",
@@ -1041,20 +1040,20 @@ states:
       refName: provisionOrderFunction
       parameters:
         order: "{{ $.order }}"
-  onError:
-  - expression: "{{ $.exceptions[?(@.name == 'MissingOrderIdException')] }}"
+  stateDataFilter:
+    dataOutputPath: "{{ $.exceptions }}"
+  transition:
+    nextState: ApplyOrder
+  onErrors:
+  - error: Missing order id
     transition:
       nextState: MissingId
-  - expression: "{{ $.exceptions[?(@.name == 'MissingOrderItemException')] }}"
+  - error: Missing order item
     transition:
       nextState: MissingItem
-  - expression: "{{ $.exceptions[?(@.name == 'MissingOrderQuantityException')] }}"
+  - error: Missing order quantity
     transition:
       nextState: MissingQuantity
-  stateDataFilter:
-    dataOutputPath: "{{ $.exceptions }}"
-  transition:
-    nextState: ApplyOrder
 - name: MissingId
   type: subflow
   workflowId: handleMissingIdExceptionWorkflow
@@ -1156,12 +1155,9 @@ In the case job submission raises a runtime error, we transition to a SubFlow st
           }
       }
       ],
-      "onError": [
+      "onErrors": [
       {
-        "expression": "{{ $.exceptions[0] }}",
-        "errorDataFilter": {
-          "dataOutputPath": "{{ $.exception }}"
-        },
+        "error": "*",
         "transition": {
           "nextState": "SubmitError"
         }
@@ -1307,10 +1303,8 @@ states:
         name: "{{ $.job.name }}"
     actionDataFilter:
       dataResultsPath: "{{ $.jobuid }}"
-  onError:
-  - expression: "{{ $.exceptions[0] }}"
-    errorDataFilter:
-      dataOutputPath: "{{ $.exception }}"
+  onErrors:
+  - error: "*"
     transition:
       nextState: SubmitError
   stateDataFilter:

roadmap/README.md

@@ -38,7 +38,7 @@ _Status description:_
 | ✔️| Update workflow start definition | [spec doc](https://github.com/cncf/wg-serverless/blob/v0.1/workflow/spec/spec.md) |
 | ✔️| Prepare github branch and docs for v0.1 | [branch](https://github.com/cncf/wg-serverless/tree/v0.1/workflow/spec) |
 
-## v0.5 (Released November 2020)
+## v0.5
 
 | Status | Description | Comments |
 | --- | --- |  --- |
@@ -72,3 +72,4 @@ _Status description:_
 | ✔️| Events definition update - add convenience way to define multiple events that share properties | [spec doc](../specification.md) |
 | ✔️| Update to function and events definitions - allow inline array def as well as uri reference to external resource | [spec doc](../specification.md) |
 | ✔️| Enforce use of OpenAPI specification in function definitions for portability | [spec doc](../specification.md) |
+| ✔️| Update workflow Error Handling | [spec doc](../specification.md) |

schema/retries.json

@@ -0,0 +1,54 @@
+{
+  "$id": "https://serverlessworkflow.org/core/retries.json",
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "description": "Serverless Workflow specification - retries schema",
+  "type": "object",
+  "retries": {
+    "type": "array",
+    "description": "Workflow Retry definitions. Define retry strategies that can be referenced in states onError definitions",
+    "items": {
+      "type": "object",
+      "$ref": "#/definitions/retrydef"
+    },
+    "minItems": 1
+  },
+  "required": [
+    "retries"
+  ],
+  "definitions": {
+    "retrydef": {
+      "type": "object",
+      "properties": {
+        "name": {
+          "type": "string",
+          "description": "Unique retry strategy name",
+          "minLength": 1
+        },
+        "delay": {
+          "type": "string",
+          "description": "Time delay between retry attempts (ISO 8601 duration format)"
+        },
+        "multiplier": {
+          "type": "string",
+          "description": "Multiplier value by which interval increases during each attempt (ISO 8601 time format)"
+        },
+        "maxAttempts": {
+          "type": ["integer","string"],
+          "minimum": 1,
+          "minLength": 0,
+          "description": "Maximum number of retry attempts."
+        },
+        "jitter": {
+          "type": ["number","string"],
+          "minimum": 0.0,
+          "maximum": 1.0,
+          "description": "If float type, maximum amount of random time added or subtracted from the delay between each retry relative to total delay (between 0.0 and 1.0). If string type, absolute maximum amount of random time added or subtracted from the delay between each retry (ISO 8601 duration format)"
+        }
+      },
+      "required": [
+        "name",
+        "maxAttempts"
+      ]
+    }
+  }
+}