Skip to content

Commit cd1fa43

Browse files
daisyfaithaumadaisyfaithauma
committed
Merge branch 'production' into universal-endpoint-path
2 parents bc1e3aa + 3db6b28 commit cd1fa43

File tree

18 files changed

+595
-28
lines changed

18 files changed

+595
-28
lines changed

‎public/__redirects‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2097,6 +2097,8 @@
20972097
/workers-ai/demos/* /workers-ai/guides/demos-architectures/:splat 301
20982098
/workers-ai/tutorials/* /workers-ai/guides/tutorials/:splat 301
20992099

2100+
# Workflows
2101+
/workflows/tutorials/ /workflows/examples 301
21002102

21012103
# Others
21022104
/logs/analytics-integrations/* /fundamentals/data-products/analytics-integrations/:splat 301

‎src/assets/images/waf/transform-rules/payload-decrypted.png‎

-78.4 KB
Loading

‎src/assets/images/waf/transform-rules/payload-logging-example.png‎

-84.4 KB
Loading

‎src/content/docs/learning-paths/workflows-course/series/index.mdx‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
pcx_content_type: navigation
3+
title: Overview
4+
sidebar:
5+
order: 4
6+
group:
7+
hideIndex: true
8+
tableOfContents: false
9+
---

‎src/content/docs/learning-paths/workflows-course/series/workflows-1.mdx‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
---
2+
pcx_content_type: navigation
3+
title: Introduction to Workflows
4+
sidebar:
5+
order: 1
6+
tableOfContents: false
7+
description: |
8+
Cloudflare Workflows provides durable execution capabilities, allowing developers to create reliable, repeatable workflows that run in the background. Workflows are designed to resume execution even if the underlying compute fails, ensuring that tasks complete eventually. They are built on top of Cloudflare Workers and handle scaling and provisioning automatically.
9+
---
10+
11+
import { Render, Tabs, TabItem, Stream, Card } from "~/components";
12+
13+
<Tabs>
14+
<TabItem label="Watch this episode">
15+
16+
Cloudflare Workflows provides durable execution capabilities, allowing developers to create reliable, repeatable workflows that run in the background. Workflows are designed to resume execution even if the underlying compute fails, ensuring that tasks complete eventually. They are built on top of Cloudflare Workers and handle scaling and provisioning automatically.
17+
18+
Workflows are triggered by events, such as Event Notifications consumed from a Queue, HTTP requests, another Worker, or even scheduled timers. Individual steps within a Workflow are designed as retriable units of work. The state is persisted between steps, allowing workflows to resume from the last successful step after failures. Workflows automatically generate metrics for each step, aiding in debugging and observability.
19+
20+
<Card>
21+
<Stream
22+
id="825b29fbf3c93d525735544f77aeb816"
23+
title="Introduction to Workflows"
24+
thumbnail="https://pub-d9bf66e086fb4b639107aa52105b49dd.r2.dev/Workflows-video-1.png"
25+
showMoreVideos={false}
26+
chapters={{
27+
"Background": "0s",
28+
"Workflows Introduction": "45s",
29+
"Punderful, an app using all of the Cloudflare primitives": "1m10s",
30+
"Vectorize": "2m35s",
31+
"Workflow code in Action": "3m0s",
32+
"Does it scale?": "7m0s",
33+
"Conclusion and next video introduction": "7m15s"
34+
}}
35+
/>
36+
37+
**Related content**
38+
39+
If you want to dive into detail, refer to the following pages:
40+
41+
- [Source code for the Punderful repository](https://github.com/craigsdennis/punderful-workflows)
42+
- [Cloudflare Workflows](/workflows/)
43+
- [Cloudflare Workers AI](/workers-ai/)
44+
45+
</Card>
46+
</TabItem>
47+
48+
<TabItem label="Step-by-step tutorial">
49+
50+
Punderful is a sample application that showcases the use of various Cloudflare primitives, including Workers, D1, Vectorize, Workers AI, and Workflows. The application displays a list of puns stored in a D1 database.
51+
52+
The homepage lists the latest puns stored in D1. The application also includes a semantic search feature powered by Vectorize. To perform a search:
53+
54+
1. Go to the Punderful search page.
55+
2. Type a search query in the "Search for a pun..." input box.
56+
3. Observe the search results appearing instantly below the search box.
57+
58+
To demonstrate adding a new pun:
59+
60+
1. Go to the Punderful creation page.
61+
2. Enter a new pun in the "Enter your pun here..." textarea.
62+
3. Observe the preview of the pun updating as you type.
63+
4. Click the "Submit Pun" button.
64+
65+
When a new pun is submitted, it needs to be indexed in Vectorize for the semantic search to work. This indexing process involves creating embeddings from the pun text. This is a task suitable for background processing using Cloudflare Workflows, avoiding delays for the user in the request-response loop.
66+
67+
### Implementing a Workflow to Process New Puns
68+
69+
A workflow is implemented to handle the background processing required when a new pun is submitted.
70+
71+
#### Triggering the Workflow
72+
73+
When a new pun is submitted via the `/api/puns` endpoint, the data is first inserted into the D1 database. Then, a new Workflow instance is created and triggered to perform the subsequent background tasks.
74+
75+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/7cec7f4bd7d6b17085cb6d6cb3e56b6a4b5b7c9d/src/index.tsx#L165)
76+
77+
In this handler, `c.env.PUBLISH.create(crypto.randomUUID(), { punId, pun: payload.pun })` creates a new instance of the workflow bound as `PUBLISH`, assigns it a unique ID, and passes the `punId` and `pun` text as the payload.
78+
79+
#### Defining the Workflow Class
80+
81+
The workflow logic is defined in a class that extends `WorkflowEntrypoint`.
82+
83+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/7cec7f4bd7d6b17085cb6d6cb3e56b6a4b5b7c9d/src/workflows/publish.ts#L12)
84+
85+
The `run` method is the entrypoint for the workflow execution. It receives the `event` containing the payload and a `step` object to define individual, durable steps.
86+
87+
#### Workflow Steps
88+
89+
Each discrete, retriable task in the workflow is defined using `await step.do()`.
90+
91+
##### Content Moderation
92+
93+
Optionally, the workflow can perform content moderation using an external service like OpenAI's moderation API if an API key is available in the environment.
94+
95+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/7cec7f4bd7d6b17085cb6d6cb3e56b6a4b5b7c9d/src/workflows/publish.ts#L16)
96+
97+
This step calls the OpenAI moderation API. If the content is flagged as inappropriate, the pun's status is updated in the database, and a `NonRetryableError` is thrown. Throwing a `NonRetryableError` prevents the workflow from retrying this step, as the content is permanently deemed inappropriate.
98+
99+
##### Creating Embeddings
100+
101+
Next, create vector embeddings for the pun text using a Workers AI model.
102+
103+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/7cec7f4bd7d6b17085cb6d6cb3e56b6a4b5b7c9d/src/workflows/publish.ts#L34)
104+
105+
This step uses the `@cf/baai/bge-large-en-v1.5` model from Workers AI to generate a vector embedding for the `pun` text. The result (the embedding vector) is returned by the step and can be used in subsequent steps. `step.do()` ensures this step will be retried if it fails, guaranteeing that embeddings are eventually created.
106+
107+
##### Categorizing the Pun
108+
109+
Optionally, use a Workers AI language model to categorize the pun.
110+
111+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/7cec7f4bd7d6b17085cb6d6cb3e56b6a4b5b7c9d/src/workflows/publish.ts#L41)
112+
113+
This step uses the `@cf/meta/llama-3.1-8b-instruct` model with a specific system prompt to generate categories for the pun. The generated categories string is returned by the step. This step also benefits from `step.do()`'s reliability.
114+
115+
##### Adding Embeddings to Vectorize
116+
117+
Insert the created pun embedding and potentially categories embedding into the Vectorize database.
118+
119+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/7cec7f4bd7d6b17085cb6d6cb3e56b6a4b5b7c9d/src/workflows/publish.ts#L78)
120+
121+
This step uses `this.env.VECTORIZE.upsert()` to add the generated embeddings and associated metadata to the Vectorize database. This makes the pun searchable semantically. `step.do()` ensures this critical indexing step is completed reliably.
122+
123+
##### Updating Database Status
124+
125+
The final step updates the status of the pun in the D1 database to indicate that it has been published and processed by the workflow.
126+
127+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/7cec7f4bd7d6b17085cb6d6cb3e56b6a4b5b7c9d/src/workflows/publish.ts#L104)
128+
129+
This step updates the `status` column in the D1 database to "published" for the corresponding pun ID. Once this step is complete, the pun is considered fully processed and ready to be displayed on the homepage.
130+
131+
#### Workflow Bindings
132+
133+
To make the `PublishWorkflow` class available to the main Worker and to provide access to necessary resources (like D1, AI, Vectorize), bindings are configured in the `wrangler.toml` file.
134+
135+
[See here](https://github.com/craigsdennis/punderful-workflows/blob/main/wrangler.toml)
136+
137+
This configuration defines a workflow named `publish`, binds it to the environment variable `PUBLISH`, and links it to the `PublishWorkflow` class in `src/index.ts`. It also shows bindings for Workers AI (`AI`) and Vectorize (`VECTORIZE`), which are accessed via `this.env` within the workflow.
138+
139+
### Vectorize for Semantic Search
140+
141+
Vectorize is a vector database used in this application to enable semantic search for puns. It stores the vector embeddings created by Workers AI. The search functionality queries this Vectorize index to find puns similar in meaning to the user's query.
142+
143+
The homepage displays recently published puns (status "published"). The detail page for a specific pun displays "Similar Puns", which are found by querying Vectorize with the embedding of the current pun.
144+
145+
### Scalability
146+
147+
Cloudflare Workers and Workflows are designed to scale automatically based on demand, handling concurrent requests and background tasks efficiently without requiring manual provisioning.
148+
149+
</TabItem>
150+
151+
<TabItem label="Series overview">
152+
<Render file="workflows-series-navigation" />
153+
</TabItem>
154+
</Tabs>

‎src/content/docs/learning-paths/workflows-course/series/workflows-2.mdx‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
---
2+
pcx_content_type: navigation
3+
title: Monitor and batch your website data
4+
sidebar:
5+
order: 2
6+
tableOfContents: false
7+
description: |
8+
Workflows can be used to process batches of data, ensuring each item in the batch goes through a defined process with reliable execution. This section demonstrates processing a batch of puns using the Punderful application as an example.
9+
10+
---
11+
12+
import { Render, Tabs, TabItem, Stream, Card } from "~/components";
13+
14+
<Tabs>
15+
<TabItem label="Watch this episode">
16+
17+
Workflows can be used to process batches of data, ensuring each item in the batch goes through a defined process with reliable execution. This section demonstrates processing a batch of puns using the Punderful application as an example.
18+
19+
<Card>
20+
<Stream
21+
id="2c36852489758c056da930e8714b6e74"
22+
title="Monitor and batch your website data"
23+
thumbnail="https://pub-d9bf66e086fb4b639107aa52105b49dd.r2.dev/Workflows-video-2.png"
24+
showMoreVideos={false}
25+
chapters={{
26+
"Introduction": "3s",
27+
"Implementing Workflows with Puns Dataset": "1m29s",
28+
"Deployment and Monitoring": "2m52s",
29+
"Admin Dashboard and Further Insights": "4m0s"
30+
}}
31+
/>
32+
33+
**Related content**
34+
35+
If you want to dive into detail, refer to the following pages:
36+
37+
- [Source code for the Punderful repository](https://github.com/craigsdennis/punderful-workflows)
38+
- [Cloudflare Workflows](/workflows/)
39+
- [Cloudflare Workers AI](/workers-ai/)
40+
41+
</Card>
42+
</TabItem>
43+
44+
<TabItem label="Step-by-step tutorial">
45+
46+
The Punderful application processes user-submitted puns by performing content moderation, creating embeddings, categorizing, and adding them to a vector store. This process is defined as a Workflow. To process a batch of existing puns (from an open-source dataset called OPun), a batch endpoint is created that iterates through the puns and triggers the defined Workflow for each one.
47+
48+
#### Batch Processing Code
49+
50+
The following code snippet shows the endpoint responsible for batch processing:
51+
52+
[See here](https://github.com/craigsdennis/punderful-workflows/tree/main/src/index.tsx#L291)
53+
54+
This code:
55+
56+
1. Fetches the list of puns from a JSON file (`puns.json`).
57+
2. Logs the number of puns being processed.
58+
3. Sets a user ID for tracking.
59+
4. Loops through each pun.
60+
5. Performs basic text cleaning on the pun.
61+
6. Inserts the pun into the database (handled by `insertPun`).
62+
7. Triggers the `PUBLISH` Workflow for each individual pun using `c.env.PUBLISH.create()`. The Workflow is given a unique ID using `crypto.randomUUID()`.
63+
64+
### Monitoring Workflow Instances via CLI
65+
66+
The Cloudflare Wrangler CLI provides commands to monitor and manage Workflows and their instances.
67+
68+
To list the available workflows associated with your account:
69+
70+
```bash
71+
npx wrangler workflows list
72+
```
73+
74+
To list the instances of a specific workflow (for example, the `publish` workflow):
75+
76+
```bash
77+
npx wrangler workflows instances list publish
78+
```
79+
80+
This command will show a list of workflow instances, their status (Queued, Running, Completed, Errored), and timestamps.
81+
82+
To view the details of a specific workflow instance, including its steps and their status, duration, and output:
83+
84+
```bash
85+
npx wrangler workflows instances describe publish <instance-id>
86+
```
87+
88+
Replace `<instance-id>` with the actual ID of a running or completed instance from the `list` command output.
89+
90+
#### Example CLI Output (Describe Instance)
91+
92+
Describing a workflow instance provides a detailed breakdown of its execution:
93+
94+
```
95+
Workflow Name: publish
96+
Instance ID: oPun-batch-aea07d75-95fa-448f-9573-6e435388eff7
97+
Version ID: 75665fce-24a1-4c83-a561-088aabc91e5f
98+
Status: Completed
99+
Trigger: API
100+
Queued: 10/24/2024, 1:43:45 AM
101+
Success: Yes
102+
Start: 10/24/2024, 1:43:45 AM
103+
End: 10/24/2024, 1:43:49 AM
104+
Duration: 4 seconds
105+
Last Successful Step: update-status-to-published-1
106+
Steps:
107+
108+
Name: content-moderation-1
109+
Type: Step
110+
Start: 10/24/2024, 1:43:45 AM
111+
End: 10/24/2024, 1:43:45 AM
112+
Duration: 0 seconds
113+
Success: Yes
114+
Output: "true"
115+
Config: {"retries":{"limit":5,"delay":1000,"backoff":"exponential"},"timeout":"10 minutes"}
116+
Attempts:
117+
Status: Completed
118+
Start Time: Oct 23, 2024 6:44:57 PM
119+
End Time: Oct 23, 2024 6:44:57 PM
120+
Wall Time: 180 ms
121+
... (additional steps like create-pun-embedding-1, categorize-pun-1, add-embeddings-to-vector-store-1, update-status-to-published-1)
122+
```
123+
124+
This output shows the status, start/end times, duration, success status, and even the output and configuration for each step within the workflow instance.
125+
126+
### Monitoring Workflow Instances via Cloudflare Dashboard
127+
128+
You can also monitor Workflows and their instances directly in the Cloudflare Dashboard.
129+
130+
This dashboard view provides a user-friendly way to observe the progress of your batch jobs, identify failed instances, and inspect the execution details of each step.
131+
132+
</TabItem>
133+
134+
<TabItem label="Series overview">
135+
<Render file="workflows-series-navigation" />
136+
</TabItem>
137+
</Tabs>

0 commit comments

Comments
 (0)