Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs-v2/pages/workflows/control-flow/_meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"index": "Overview",
"ifelse": "Branching - If/Else",
"switch": "Branching - Switch",
"parallel": "Branching - Parallel",
"delay": "Delay",
"filter": "Filter",
"end-workflow": "End Workflow"
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
99 changes: 99 additions & 0 deletions docs-v2/pages/workflows/control-flow/parallel.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
import Callout from '@/components/Callout'
import VideoPlayer from "@/components/VideoPlayer";
import { Steps } from 'nextra/components'

# Parallel

## Overview

**Parallel** is multi-path branching operator. It allows you to create multiple execution branches with optional filtering rules and Pipedream will execute **all** matching branches. Unlike [Switch](./switch) and [If/Else](./ifelse), the order in which rules are defined will not affect the path of execution.

![Parallel](./images/parallel/parallel.png)

## Capabilities

- Create non-linear workflows that execute steps in parallel branches
- Define when branches run — always, conditionally or never (to disable a branch)
- Merge and continue execution in the parent flow after the branching operation

<Callout type="info">
The Parallel operator is a control flow **Block** with **start** and **end** phases. [Learn more about Blocks](./#blocks).
</Callout>

### Add Parallel operator to workflow

Select **Parallel** from the **Control Flow** section of the step selector:

![add parallel block](./images/parallel/add_parallel_block.png)

### Create Branches

To create new branches, click the `+` button:

![add branch](./images/parallel/add_branch.png)

### Rename Branches

Edit the branch's nameslug on the canvas or in the right pane after selecting the **Start** phase of the parallel block. The nameslug communicates the branch's purpose and affects workflow execution—the end phase exports an object, with each key corresponding to a branch name.

![rename branch](./images/parallel/rename_branch.png)

### Export Data to the Parent Flow

You can export data from a parallel operation and continue execution in the parent flow.
- The parallel block exports data as a JSON object
- Branch exports are assigned to a key corresponding to the branch name slug (in the object exported from the block)
- Only the exports from the last step of each executed branch are included in the parallel block's return value
- To preview the exported data, test the **End** phase of the parallel block

Comment on lines +41 to +48
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add a code example for data exports.

Consider adding a simple code example showing the structure of exported data to help users better understand how the data is organized.

### Export Data to the Parent Flow

You can export data from a parallel operation and continue execution in the parent flow. 
- The parallel block exports data as a JSON object
- Branch exports are assigned to a key corresponding to the branch name slug (in the object exported from the block) 
- Only the exports from the last step of each executed branch are included in the parallel block's return value
- To preview the exported data, test the **End** phase of the parallel block

+For example, if you have branches named "process_orders" and "send_notifications", the exported data might look like:
+```json
+{
+  "process_orders": {
+    "processed_count": 42,
+    "status": "success"
+  },
+  "send_notifications": {
+    "sent_count": 10,
+    "failed_count": 0
+  }
+}
+```

### Beta Limitations

Workflow queue settings (concurrency, execution rate) may not work as expected with workflows using the parallel operator.

## Getting Started

<Steps>

### Generate a test event

Add a trigger and generate an event to help you build and test your workflow:

![trigger.gif](./images/parallel/01_trigger.gif)

### Add the Parallel control flow block

Click the + button to add a step to the canvas and select Parallel from the Control Flow section on the right. You can optionally add or remove branches and configure conditions defining when each branch should run.

![add parallel.gif](./images/parallel/02_add_parallel.gif)

### Test to identify the execution path(s)

Test the **Start** phase to identify which branches will execute for the current event.

![configure and test.gif](./images/parallel/03_configure_and_test.gif)

### Add steps to branches

Add steps to the branches. These steps will be executed in parallel when the workflow runs.

![add steps.gif](./images/parallel/04_add_steps.gif)

### Optionally merge and continue to the parent flow

Test the **End** phase to export the results of the last step of each branch that was executed. This makes data from the branches available to reference in the parent flow.

![export_data_to_parent](./images/parallel/05_test_end_phase.gif)

### Use exports in parent flow

Optionally add steps after the parallel block and use data from individual branches by referencing the return value of the **End** phase.

![use exports in parent flow](./images/parallel/06_use_exports_in_parent_flow.gif)

### Deploy and test the live workflow

Deploy the workflow and trigger it to inspect the executions.

![Inspect](./images/parallel/07_deploy_and_run.gif)

</Steps>
Loading