Skip to content

Commit 117e076

Browse files
committed
docs: add Windmill documentation
1 parent 30d0148 commit 117e076

File tree

7 files changed

+231
-0
lines changed

7 files changed

+231
-0
lines changed
101 KB
Loading
25.8 KB
Loading
100 KB
Loading
67.5 KB
Loading
160 KB
Loading
176 KB
Loading
Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
---
2+
title: Windmill integration
3+
description: Use Windmill to run Apify Actors and Tasks, react to Apify events via webhooks or polling, and move data between Apify and other services with Windmill scripts and flows.
4+
sidebar_label: Windmill
5+
sidebar_position: 8
6+
slug: /integrations/windmill
7+
---
8+
9+
**Use Windmill to run Apify Actors and Tasks, react to Apify events via webhooks or polling, and move data between Apify and other services with Windmill scripts and flows.**
10+
11+
---
12+
13+
[Windmill](https://www.windmill.dev/) is an open-source automation platform for building scripts and flows that connect your tools and data. With the Apify integration for Windmill, you can run Actors and Tasks, scrape websites, extract data from storage, and trigger workflows based on Apify events.
14+
15+
This guide shows you how to install the Apify package, set up authentication, and create automated workflows that integrate with Apify.
16+
17+
## Prerequisites
18+
19+
Before you begin, make sure you have:
20+
21+
- An [Apify account](https://console.apify.com/)
22+
- A [Windmill instance](https://www.windmill.dev/docs/getting_started/how_to_use_windmill) (self-hosted or cloud)
23+
24+
## Install the Apify integration
25+
26+
The Apify integration provides scripts, flows, and resources that will be available through the Windmill Hub.
27+
28+
<!-- SCREENSHOT: Windmill Hub showing available integrations including Apify alongside with other integrations -->
29+
30+
### Step 1: Import Apify scripts from Windmill Hub
31+
32+
You can import Apify integration scripts into your flows from the Windmill Hub, regardless of whether you're using Windmill Cloud or a self-hosted instance. The following components will be available:
33+
34+
![Apify Auth](../images/windmill-install-hub.png)
35+
36+
**Scripts (Actions):**
37+
38+
- Run Actor
39+
- Run Task
40+
- Scrape Single URL
41+
- Get Dataset Items
42+
- Get Key-Value Store
43+
44+
**Scripts (Triggers):**
45+
46+
- Polling-based Actor Run Trigger
47+
- Polling-based Task Trigger
48+
49+
**Flows (Triggers):**
50+
51+
- Webhook-based Actor Trigger
52+
- Webhook-based Task Trigger
53+
54+
**Resources:**
55+
56+
- Apify Auth Resource (for storing your API token)
57+
- Webhook Config Resource (optional, for managing webhook-based triggers)
58+
59+
<!-- SCREENSHOT: Windmill Hub showing available integrations including Apify -->
60+
<!-- SCREENSHOT: Importing Apify scripts from Windmill Hub -->
61+
62+
### Step 2: Authentication
63+
64+
You can provide the token to scripts via a **Windmill Resource**. Create it either in the **Resources** tab or directly from a script.
65+
66+
#### Option A — Create in the Resources tab
67+
68+
1. Open **Resources****New Resource**.
69+
2. Select `apify_api_key` resource type.
70+
3. Name it (e.g., `apify_token`) and paste your Apify API token.
71+
4. Save, then reference this resource in your scripts/flows.
72+
73+
![Apify Auth](../images/windmill-install-auth-resource-tab.png)
74+
75+
#### Option B — Create/bind from a script
76+
77+
1. Open the script in Windmill UI.
78+
2. Add a secret input parameter (e.g., `apify_token`) .
79+
3. Bind it to the resource you created (or create a new one inline).
80+
4. Use the bound token inside the script to call Apify.
81+
82+
![Apify Auth](../images/windmill-install-auth-script.png)
83+
84+
## Create your first workflow
85+
86+
Let's create a simple workflow that runs an Actor and fetches its results.
87+
88+
### Step 1: Create a new flow
89+
90+
1. In the Windmill UI, click **New Flow**.
91+
2. Give your flow a descriptive name (e.g., "Run Actor and Get Results").
92+
93+
### Step 2: Add the Run Actor script
94+
95+
1. Click **Add Step** and search for "Run Actor".
96+
2. Select the **Run Actor** script.
97+
3. Configure the inputs:
98+
- **Apify Auth**: Select your `apify_token` resource
99+
- **Actor ID or Slug**: Enter the Actor you want to run
100+
- **Input**: JSON input for the Actor (optional)
101+
- **Wait for Finish**: Set to `true` to wait for completion
102+
- **Memory**: Memory allocation in MB (optional)
103+
- **Timeout**: Timeout in seconds (optional)
104+
105+
![Apify Auth](../images/windmill-flow-run-actor.png)
106+
107+
### Step 3: Add the Get Dataset Items script
108+
109+
1. Add another step and search for "Get Dataset Items".
110+
2. Configure the inputs:
111+
- **Apify Auth**: Select your `apify_token` resource
112+
- **Dataset ID**: Use the `defaultDatasetId` from the previous step
113+
- **Limit**: Number of items to retrieve (optional)
114+
115+
![Apify Auth](../images/windmill-flow-get-dataset.png)
116+
117+
### Step 4: Test and run
118+
119+
With the flow crated and scripts linked we can test and run it.
120+
121+
![Apify Auth](../images/windmill-flow-linked.png)
122+
123+
1. Click **Test** to run the flow with sample data.
124+
2. Review the results and ensure both steps completed successfully.
125+
3. Save and activate your flow.
126+
127+
<!-- SCREENSHOT: Flow execution results showing successful completion -->
128+
129+
## Use webhooks to trigger workflows
130+
131+
Windmill provides webhook-based triggers that can automatically start workflows when Apify events occur.
132+
133+
### Step 1: Fork the example flow
134+
135+
1. Fork the **Apify example flow with webhook trigger** from Windmill’s templates.
136+
2. In the forked flow, add a trigger of type **Webhook**.
137+
3. Create a **Webhook-specific Token**.
138+
4. Copy both the **token** and the **webhook URL** to your clipboard.
139+
140+
<!-- SCREENSHOT: Webhook trigger configuration -->
141+
<!-- SCREENSHOT: Webhook URL and token display -->
142+
143+
### Step 2: Configure the webhook creation script
144+
145+
1. In the flow, open the script that creates the Apify webhook.
146+
2. Set the **Apify Auth resource** (`apify_token`).
147+
3. Create a new **webhook config resource** with:
148+
149+
- **Actor ID**: the Apify Actor you want to monitor
150+
- **Event Types**: the events that should trigger the flow
151+
- Other required values
152+
153+
4. **IMPORTANT**: Test-run this script to **create the webhook in Apify** for the Actor.
154+
155+
<!-- SCREENSHOT: Apify webhook creation script configuration -->
156+
157+
### Step 3: Adjust the flow logic
158+
159+
1. Replace the default logic in the flow with your desired actions:
160+
- Example: persist results into a Google Sheet.
161+
2. If needed, use the provided **helper script** to “mold” the webhook payload into the correct shape for the Google Sheets operation.
162+
163+
<!-- SCREENSHOT: Flow logic customization -->
164+
<!-- SCREENSHOT: Payload molding helper -->
165+
166+
### Step 4: Test the flow
167+
168+
1. Deploy the flow so it can be triggered by the webhook.
169+
2. Run the Actor in Apify that the webhook is registered for.
170+
3. Check the **flow runs** in Windmill to verify that the test run was registered and the logic executed.
171+
172+
<!-- SCREENSHOT: Flow execution triggered by webhook -->
173+
<!-- SCREENSHOT: Execution logs showing webhook payload -->
174+
175+
## Available operations
176+
177+
The Apify integration provides several operations you can use in your Windmill workflows.
178+
179+
### Actions (Scripts)
180+
181+
**Run Actor**
182+
183+
- Starts an Actor with optional input and configuration
184+
- Can wait for completion or run asynchronously
185+
- Returns run metadata including dataset ID
186+
187+
**Run Task**
188+
189+
- Executes a predefined Actor task
190+
- Similar to Run Actor but uses task configuration
191+
- Ideal for recurring operations
192+
193+
**Scrape Single URL**
194+
195+
- Runs a lightweight scraper for a single webpage
196+
- Returns content as text, markdown, and HTML
197+
- Perfect for quick content extraction
198+
199+
**Get Dataset Items**
200+
201+
- Retrieves items from a dataset
202+
- Can filter by dataset ID or last run
203+
- Supports pagination and item limits
204+
205+
**Get Key-Value Store**
206+
207+
- Reads values from key-value stores
208+
- Can retrieve specific keys or list all keys
209+
- Useful for configuration and state management
210+
211+
### Triggers
212+
213+
**Webhook-based Triggers**
214+
215+
- **Actor Webhook Trigger**: Responds to Actor run events
216+
- **Task Webhook Trigger**: Responds to task run events
217+
- Real-time event processing with full payload data
218+
219+
**Polling-based Triggers**
220+
221+
- **Actor Run Polling Trigger**: Periodically checks for new Actor runs
222+
- **Task Run Polling Trigger**: Periodically checks for new task runs
223+
- Fallback option when webhooks aren't available
224+
225+
## Resources
226+
227+
- [Windmill Documentation](https://www.windmill.dev/docs)
228+
- [Windmill Local Development](https://www.windmill.dev/docs/advanced/local_development)
229+
- [Apify API Documentation](https://docs.apify.com)
230+
- [Apify Webhooks](https://docs.apify.com/webhooks)
231+
- [Apify Actors & Tasks](https://docs.apify.com/actors)

0 commit comments

Comments
 (0)