docs: describe data flow visually

Signed-off-by: Matthias Pichler <[email protected]>

Author: matthias-pichler

dsl-reference.md

@@ -243,7 +243,7 @@ The Serverless Workflow DSL defines a list of [tasks](#task) that **must be** su
 
 | Name | Type | Required | Description|
 |:--|:---:|:---:|:---|
-| if | `string` | `no` | A [`runtime expression`](dsl.md#runtime-expressions), if any, used to determine whether or not the task should be run.<br>The task is considered skipped if not run. The runtime expression evaluates in the context of the filtered task input (i.e. result of the `input.from` expression, if any) |
+| if | `string` | `no` | A [`runtime expression`](dsl.md#runtime-expressions), if any, used to determine whether or not the task should be run.<br>The task is considered skipped if not run. |
 | input | [`input`](#input) | `no` | An object used to customize the task's input and to document its schema, if any. |
 | output | [`output`](#output) | `no` | An object used to customize the task's output and to document its schema, if any. |
 | export | [`export`](#export) | `no` | An object used to customize the content of the workflow context. | 

dsl.md

@@ -149,37 +149,93 @@ In Serverless Workflow DSL, data flow management is crucial to ensure that the r
 Here's how data flows through a workflow based on various filtering stages:
 
 1. **Filter Workflow Input**
-Before the workflow starts, the input data provided to the workflow can be filtered to ensure only relevant data is passed into the workflow context. This step allows the workflow to start with a clean and focused dataset, reducing potential overhead and complexity in subsequent tasks.
+Before the workflow starts, the input data provided to the workflow can be filtered to ensure only relevant data is passed into the workflow context. This can be done using the top level `input.from` expression. It evaluates on the raw workflow input and defaults to the identity expression which leaves the input unchanged. This step allows the workflow to start with a clean and focused dataset, reducing potential overhead and complexity in subsequent tasks. The result of this expression will set as the initial value for the `$context` runtime expression argument and be passed to the first task.
 
 *Example: If the workflow receives a JSON object as input, a filter can be applied to remove unnecessary fields and retain only those that are required for the workflow's execution.*
 
 2. **Filter First Task Input**
-The input data for the first task can be filtered to match the specific requirements of that task. This ensures that the first task receives only the necessary data it needs to perform its operations.
+The input data for the first task can be filtered to match the specific requirements of that task. This ensures that the first task receives only the necessary data it needs to perform its operations. This can be done using the task's `input.from` expression. It evaluates on the filtered workflow input and defaults to the identity expression which leaves the input unchanged. The result of this expression will be set as the `$input` runtime expression argument and be passed to the task. Any runtime expressions used within the task definition will be evaluated using this filtered input.
 
 *Example: If the first task is a function call that only needs a subset of the workflow input, a filter can be applied to provide only those fields needed for the function to execute.*
 
 3. **Filter First Task Output**
-After the first task completes, its output can be filtered before passing it to the next task or storing it in the workflow context. This helps in managing the data flow and keeping the context clean by removing any unnecessary data produced by the task.
+After the first task completes, its output can be filtered before passing it to the next task or storing it in the workflow context. Filtering is performed using the `output.as` runtime expression. It evaluates on the raw task output and defaults to the identity expression which leaves the output unchanged. Its result will be the input to the next task. To update the context one uses the `export.as` runtime expression. It evaluates on the raw output and defaults to the expression that returns the existing context. The result of this runtime expression replaces the workflows current context and the content of the `$context`runtime expression argument. This helps in managing the data flow and keeping the context clean by removing any unnecessary data produced by the task.
 
 *Example: If the first task returns a large dataset, a filter can be applied to retain only the relevant results needed for subsequent tasks.*
 
 4. **Filter Last Task Input**
-Before the last task in the workflow executes, its input data can be filtered to ensure it receives only the necessary information. This step is crucial for ensuring that the final task has all the required data to complete the workflow successfully.
+Before the last task in the workflow executes, its input data can be filtered to ensure it receives only the necessary information. This can be done using the task's `input.from` expression. It evaluates on the filtered workflow input and defaults to the identity expression which leaves the input unchanged. The result of this expression will be set as the `$input` runtime expression argument and be passed to the task. Any runtime expressions used within the task definition will be evaluated using this filtered input. This step is crucial for ensuring that the final task has all the required data to complete the workflow successfully.
 
 *Example: If the last task involves generating a report, the input filter can ensure that only the data required for the report generation is passed to the task.*
 
 5. **Filter Last Task Output**
-After the last task completes, its output can be filtered before it is considered as the workflow output. This ensures that the workflow produces a clean and relevant output, free from any extraneous data that might have been generated during the task execution.
+After the last task completes, its output can be filtered before it is considered as the workflow output. Filtering is performed using the `output.as` runtime expression. It evaluates on the raw task output and defaults to the identity expression which leaves the output unchanged. Its result will be passes to the workflow `output.as` runtime expression. This ensures that the workflow produces a clean and relevant output, free from any extraneous data that might have been generated during the task execution.
 
 *Example: If the last task outputs various statistics, a filter can be applied to retain only the key metrics that are relevant to the stakeholders.*
 
 6. **Filter Workflow Output**
-Finally, the overall workflow output can be filtered before it is returned to the caller or stored. This step ensures that the final output of the workflow is concise and relevant, containing only the necessary information that needs to be communicated or recorded.
+Finally, the overall workflow output can be filtered before it is returned to the caller or stored. Filtering is performed using the `output.as` runtime expression. It evaluates on the last task's output and defaults to the identity expression which leaves the output unchanged. This step ensures that the final output of the workflow is concise and relevant, containing only the necessary information that needs to be communicated or recorded.
 
 *Example: If the workflow's final output is a summary report, a filter can ensure that the report contains only the most important summaries and conclusions, excluding any intermediate data.*
 
 By applying filters at these strategic points, Serverless Workflow DSL ensures that data flows through the workflow in a controlled and efficient manner, maintaining clarity and relevance at each stage of execution. This approach helps in managing complex workflows and ensures that each task operates with the precise data it requires, leading to more predictable and reliable workflow outcomes.
 
+Visually this can be represented as follows:
+
+```mermaid
+flowchart TD
+
+  initial_context_arg([<code>$context</code>])
+  context_arg([<code>$context</code>])
+  input_arg([<code>$input</code>])
+
+  workflow_raw_input{{Raw Workflow Input}}
+  workflow_input_from[Workflow: <code>input.from</code>]
+  workflow_filtered_input{{Filtered Workflow Input}}
+
+  task_raw_input{{Raw Task Input}}
+  task_input_from[Task: <code>input.from</code>]
+  task_filtered_input{{Filtered Task Input}}
+  task_definition[Task definition]
+  task_raw_output{{Raw Task output}}
+  task_output_as[Task: <code>output.as</code>]
+  task_filtered_output{{Filtered Task output}}
+  task_export_as[Task: <code>export.as</code>]
+  
+  workflow_raw_output{{Raw Workflow Output}}
+  workflow_output_as[Workflow: <code>output.as</code>]
+  workflow_filtered_output{{Filtered Workflow Output}}
+
+  workflow_raw_input --> workflow_input_from
+  workflow_input_from -- Produces --> workflow_filtered_input
+  workflow_filtered_input -- Set as --> initial_context_arg
+  workflow_filtered_input -- Passed to --> task_raw_input
+
+  subgraph Task
+
+    task_raw_input -- Passed to --> task_input_from
+    task_input_from -- Produces --> task_filtered_input
+    task_filtered_input -- Set as --> input_arg
+    task_filtered_input -- Passed to --> task_definition
+
+    task_definition -- Execution produces --> task_raw_output
+    task_raw_output -- Passed to --> task_output_as
+    task_output_as -- Produces --> task_filtered_output
+    task_filtered_output -- Passed to --> task_export_as
+  end
+
+  task_filtered_output -- Passed as raw input to --> next_task
+
+  subgraph next_task [Next Task]
+  end
+
+  task_export_as -- Result set as --> context_arg
+
+  next_task -- Filtered output becomes --> workflow_raw_output
+  workflow_raw_output -- Passed to --> workflow_output_as
+  workflow_output_as -- Produces --> workflow_filtered_output
+```
+
 ### Runtime Expressions
 
 Runtime expressions serve as dynamic elements that enable flexible and adaptable workflow behaviors. These expressions provide a means to evaluate conditions, transform data, and make decisions during the execution of workflows.