Skip to content

Commit 2a53593

Browse files
committed
Tidy up studios -> slack example docs, some other tidying
1 parent e00d4ef commit 2a53593

File tree

8 files changed

+138
-11
lines changed

8 files changed

+138
-11
lines changed

.github/workflows/deploy-docs.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,6 @@ jobs:
3232
restore-keys: |
3333
mkdocs-material-
3434
35-
- run: pip install mkdocs-material pymdown-extensions
35+
- run: pip install mkdocs-material pymdown-extensions mkdocs-glightbox
3636

3737
- run: mkdocs gh-deploy --force

docs/examples/03-auto-resume-on-failure.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
`examples/03 - Auto-resume on workflow failure.json`
44

5+
![03 - Auto-resume on workflow failure.json](../img/example_flow_03.png)
6+
57
This workflow demonstrates how to automatically resume a failed Nextflow workflow using the workflow ID from the failed run. This pattern is useful for recovering from transient errors without re-running successfully completed tasks.
68

79
The flow implements an automatic resume pattern:

docs/examples/04-rnaseq-differential-abundance.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,8 @@ This workflow comes **pre-configured with test data** and works out of the box!
2525

2626
**Test files (GitHub-hosted):**
2727

28-
- Samplesheet: `examples/data/rnaseq-tests-datasets.csv` - includes `sample`, `fastq_1`, `fastq_2`, `strandedness`, `treatment`, and `replicate` columns
29-
- Contrasts: `examples/data/rnaseq-test-contrasts.csv` - defines comparisons like WT vs RAP1_UNINDUCED
28+
- Samplesheet: `docs/examples/data/rnaseq-tests-datasets.csv` - includes `sample`, `fastq_1`, `fastq_2`, `strandedness`, `treatment`, and `replicate` columns
29+
- Contrasts: `docs/examples/data/rnaseq-test-contrasts.csv` - defines comparisons like WT vs RAP1_UNINDUCED
3030
- Genome: `R64-1-1` (S. cerevisiae from iGenomes)
3131

3232
**To use your own data**, edit the function node:

docs/examples/05-studio-slack-webhook.md

Lines changed: 126 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,26 @@
11
---
22
title: Studio + Slack webhook
3+
glightbox: true
34
---
45

56
# Pipeline → Add Studio → Slack webhook
67

78
`examples/05 - Studio on run fail + Slack webhook.json`
89

9-
![05 - Studio on run fail + Slack webhook.json](../img/example_flow_03.png)
10+
![05 - Studio on run fail + Slack webhook.json](../img/example_flow_05.png)
11+
12+
!!! example "In development"
13+
14+
This example is still in development and needs some work:
15+
16+
- Pre-configure the mounting of the work directory from the failed task into the studio automatically.
17+
- Use a _Monitor Studio_ node to wait until the studio is running before sending the Slack notification.
18+
19+
Contributions welcome!
20+
21+
This workflow demonstrates intelligent failure handling with automated debugging setup and Slack notifications. It launches a workflow, monitors its execution, and sends contextualized Slack messages with direct links - either to the successful run or to a freshly provisioned Seqera Studio for debugging failures.
22+
23+
## Overview
1024

1125
This workflow has three parts to it:
1226

@@ -24,19 +38,123 @@ If all goes well, you'll get messages in Slack that look like this:
2438

2539
Clicking "Open link" will take you directly to a Studio running VS Code, with your pipeline's work directory mounted and ready to debug.
2640

41+
## How it Works
42+
43+
### 1. Workflow Launch & Monitoring
44+
45+
The flow starts with a manual inject node (click trigger) that launches a workflow via the **Launch workflow** node. The workflow ID is automatically passed to the **Monitor workflow** node, which polls the workflow status every 5 seconds.
46+
47+
The monitor node has three outputs:
48+
49+
- **Output 1** (Active): Fires while workflow is running (unused in this example)
50+
- **Output 2** (Succeeded): Fires when workflow completes successfully
51+
- **Output 3** (Failed): Fires when workflow fails, cancelled, or errors
52+
53+
### 2. Success Path → Direct Link
54+
55+
When the workflow succeeds, the message flows to the **"Set webhook payload on success"** change node, which uses JSONata to construct a Slack message:
56+
57+
```json
58+
{
59+
"msg": "Workflow " & payload.workflow.projectName & " (" & payload.workflow.runName & "): Status " & payload.workflow.status & "\n\n Click the button below to view the workflow:",
60+
"button_link": "https://cloud.seqera.io/orgs/" & payload.orgName & "/workspaces/" & payload.workspaceName & "/watch/" & payload.workflow.id
61+
}
62+
```
63+
64+
This creates a deep link directly to the workflow run page in Seqera Platform.
65+
66+
### 3. Failure Path → Studio Creation
67+
68+
When the workflow fails, a more sophisticated sequence occurs:
69+
70+
1. **Save workflow context:**
71+
72+
- The **"Save payload on fail"** change node stores the workflow output in flow context (`flow.workflow_output`) so it can be referenced later, after the Studio is created.
73+
74+
1. **Create debugging Studio:**
75+
76+
- The **Add Studio** node provisions a new Seqera Studio with:
77+
- **Name**: Generated dynamically using JSONata: `"debug-pipeline-run-" & payload.workflow.id`
78+
- **Container**: VS Code (`public.cr.seqera.io/platform/data-studio-vscode:1.93.1-0.8`)
79+
- **Resources**: 2 CPU, 8GB memory
80+
- **Auto-start**: Enabled (Studio starts immediately)
81+
- **Mount Data**: Configure this to mount the Data Link containing your workflow's work directory
82+
83+
1. **Build Slack message with Studio link:**
84+
85+
- The **"Set webhook payload"** change node retrieves the saved workflow info from flow context and combines it with the newly created Studio URL:
86+
87+
```json
88+
{
89+
"msg": "Workflow " & $flowContext("workflow_output.workflow.projectName") & " (" & $flowContext("workflow_output.workflow.runName") & "): Status " & $flowContext("workflow_output.workflow.status") & "\n\n Click the button below to open a Seqera Studios session to debug:",
90+
"button_link": payload.studio.studioUrl
91+
}
92+
```
93+
94+
### 4. Slack Webhook Notification
95+
96+
Both success and failure paths converge at the **Slack webhook** HTTP request node, which POSTs the payload to your Slack webhook URL. The Slack workflow receives the `msg` and `button_link` fields and displays them as a formatted message with an actionable button.
97+
2798
## Setup
2899

29100
This workflow needs some setup:
30101

102+
### 1. Configure Seqera Nodes
103+
31104
- All Seqera nodes need a Seqera configuration to be assigned
32-
- Data Links, Pipelines, and Compute Environments need to be added to nodes
33-
- Slack workflow needs to be created and webook URL entered.
105+
- Set the **Launchpad** name in the Launch workflow node (or provide pipeline URL and compute environment)
106+
- Set the **Compute Environment ID** in the Add Studio node
107+
- Configure **Mount Data** in the Add Studio node to mount the Data Link containing your workflow work directory
108+
109+
!!! example "Help wanted"
110+
111+
This _Configure Mount Data_ is the bit that we should be able to do a better job of including in the example 👆🏻
112+
113+
### 2. Create Slack Workflow
114+
115+
To enable automated Slack notifications, you need to create a Slack Workflow with a webhook trigger:
116+
117+
1. Create a new workflow in Slack and select _"From a webhook"_ as the trigger:
118+
![Create Slack workflow](../img/example_flow_05_slack_workflow_create.png)
119+
120+
1. Configure the workflow with two input variables and a button action:
121+
122+
![Slack workflow](../img/example_flow_05_slack_workflow.png)
123+
124+
- Your Slack workflow should have:
125+
126+
- **Variable 1**: `msg` (text) - The notification message
127+
- **Variable 2**: `button_link` (text) - The URL for the action button
128+
- **Action**: Send message with button linking to `button_link`
129+
130+
1. Copy the webhook URL provided by Slack.
131+
132+
### 3. Configure Webhook URL
133+
134+
Paste the Slack webhook URL into the **"Slack webhook"** HTTP request node in the Node-RED flow. The request should be configured as:
135+
136+
- **Method**: `POST`
137+
- **URL**: Your Slack webhook URL
138+
- **Return**: `a UTF-8 string`
139+
140+
## Testing
141+
142+
To test the workflow:
34143

35-
To make the automated Slack message, first you need to create a Slack Workflow.
36-
Selelct _"From a webhook"_ as the trigger:
144+
1. **Deploy** the flow in Node-RED
145+
2. Click the blue square on the inject node to manually trigger
146+
3. Wait for the workflow to complete
147+
4. Check Slack for the notification:
148+
- **Success**: Message with link to workflow run page
149+
- **Failure**: Message with link to a running VS Code Studio session
37150

38-
![Create Slack workflow](../img/example_flow_05_slack_workflow_create.png)
151+
## Customization
39152

40-
Then configure with some input variables and a button, and copy the webhook URL. You'll need this later!
153+
This pattern can be adapted for other use cases:
41154

42-
![Slack workflow](../img/example_flow_05_slack_workflow.png)
155+
- **Different notification systems**: Replace the Slack webhook with Discord, Microsoft Teams, PagerDuty, etc.
156+
- **Studio container**: Change the container image to RStudio, JupyterLab, or custom images
157+
- **Conditional Studio creation**: Add a switch node to only create Studios for specific failure types
158+
- **Studio lifecycle**: Add a lifespan to auto-terminate Studios after a set period (configure in Add Studio node)
159+
- **Mount multiple Data Links**: Add additional mount points to access reference data or shared resources
160+
- **Email notifications**: Use an email node instead of webhooks for notifications

docs/examples/index.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@ To jump-start your development, take a look at the examples that come with the e
1616

1717
## Available Examples
1818

19+
!!! info "More examples wanted"
20+
21+
Do you have a workflow that you think would be interesting to other people?
22+
We'd love to build up our library of examples! Please open a PR! ✨
23+
1924
1. [**Simple launch & monitor**](01-simple-launch-monitor.md) - The most basic workflow demonstrating core Launch and Monitor nodes
2025
2. [**Launch on file upload**](02-launch-on-file-upload.md) - Automatically launch pipelines when files are detected using Data Link polling
2126
3. [**Auto-resume on workflow failure**](03-auto-resume-on-failure.md) - Automatically resume failed workflows to recover from transient errors

docs/img/example_flow_03.png

16.2 KB
Loading

docs/img/example_flow_05.png

133 KB
Loading

mkdocs.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,8 @@ extra_css:
7979

8080
plugins:
8181
- search
82+
- glightbox:
83+
manual: true
8284

8385
markdown_extensions:
8486
- admonition

0 commit comments

Comments
 (0)