diff --git a/src/content/changelogs/workflows.yaml b/src/content/changelogs/workflows.yaml new file mode 100644 index 000000000000000..ffdc7c5419ad54f --- /dev/null +++ b/src/content/changelogs/workflows.yaml @@ -0,0 +1,15 @@ +--- +link: "/workflows/reference/changelog/" +productName: Workflows +productLink: "/workflows/" +productArea: Developer platform +productAreaLink: /workers/platform/changelog/platform/ +entries: + - publish_date: "2024-10-23" + title: "Workflows is now in public beta!" + description: |- + Workflows, a new product for building reliable, multi-step workflows using Cloudflare Workers, is now in public beta. The public beta is available to any user with a [free or paid Workers plan](/workers/platform/pricing/). + + A Workflow allows you to define multiple, independent steps that encapsulate errors, automatically retry, persist state, and can run for seconds, minutes, hours or even days. A Workflow can be useful for post-processing data from R2 buckets before querying it, automating a Workers AI RAG pipeline, or managing user signup flows and lifecycle emails. + + You can learn more about Workflows in [our announcement blog](https://blog.cloudflare.com/building-workflows-durable-execution-on-workers/), or start building in our [get started guide](/workflows/get-started/guide/). diff --git a/src/content/docs/workers/configuration/cron-triggers.mdx b/src/content/docs/workers/configuration/cron-triggers.mdx index f905ad44ea8b5f1..8fd35c98942abec 100644 --- a/src/content/docs/workers/configuration/cron-triggers.mdx +++ b/src/content/docs/workers/configuration/cron-triggers.mdx @@ -11,6 +11,12 @@ Cron Triggers allow users to map a cron expression to a Worker using a [`schedul Cron Triggers are ideal for running periodic jobs, such as for maintenance or calling third-party APIs to collect up-to-date data. Workers scheduled by Cron Triggers will run on underutilized machines to make the best use of Cloudflare's capacity and route traffic efficiently. +:::note + +Cron Triggers can also be combined with [Workflows](/workflows/) to trigger multi-step, long-running tasks. You can [bind to a Workflow](/workflows/build/workers-api/) from directly from your Cron Trigger to execute a Workflow on a schedule. + +::: + Cron Triggers execute on UTC time. ## Add a Cron Trigger diff --git a/src/content/docs/workers/runtime-apis/bindings/workflows.mdx b/src/content/docs/workers/runtime-apis/bindings/workflows.mdx new file mode 100644 index 000000000000000..4410f72314e7424 --- /dev/null +++ b/src/content/docs/workers/runtime-apis/bindings/workflows.mdx @@ -0,0 +1,10 @@ +--- +pcx_content_type: navigation +title: Workflows +external_link: /workflows/ +head: [] +description: APIs available in Cloudflare Workers to interact with + Workflows. Workflows allow you to build durable, multi-step applications + using Workers. + +--- diff --git a/src/content/docs/workers/wrangler/commands.mdx b/src/content/docs/workers/wrangler/commands.mdx index 3346194f55b6e1c..a36edf5d3cdc38e 100644 --- a/src/content/docs/workers/wrangler/commands.mdx +++ b/src/content/docs/workers/wrangler/commands.mdx @@ -28,6 +28,7 @@ Wrangler offers a number of commands to manage your Cloudflare Workers. - [`r2 object`](#r2-object) - Manage Workers R2 objects. - [`secret`](#secret) - Manage the secret variables for a Worker. - [`secret:bulk`](#secretbulk) - Manage multiple secret variables for a Worker. +- [`workflows`](#workflows) - Manage and configure Workflows. - [`tail`](#tail) - Start a session to livestream logs from a deployed Worker. - [`pages`](#pages) - Configure Cloudflare Pages. - [`queues`](#queues) - Configure Workers Queues. @@ -138,10 +139,10 @@ wrangler docs [] ## `init` -:::note +:::note -The `init` command will be removed in a future version. Please use `npm create cloudflare@latest` instead. +The `init` command will be removed in a future version. Please use `npm create cloudflare@latest` instead. ::: @@ -1244,6 +1245,154 @@ Finished processing secrets JSON file: 🚨 1 secrets failed to upload ``` +## `workflows` + +:::note + +The `wrangler workflows` command requires Wrangler version `3.83.0` or greater. Use `npx wrangler@latest` to always use the latest Wrangler version when invoking commands. + +::: + +Manage and configure [Workflows](/workflows/). + +### `list` + +Lists the registered Workflows for this account. + +```sh +wrangler workflows list +``` + +- `--page` + - Show a specific page from the listing. You can configure page size using "per-page". +- `--per-page` + - Configure the maximum number of Workflows to show per page. + +### `instances` + +Manage and interact with specific instances of a Workflow. + +### `instances list` + +List Workflow instances. + +```sh +wrangler workflows instances list [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. + +### `instances describe` + +Describe a specific instance of a Workflow, including its current status, any persisted state, and per-step outputs. + +```sh +wrangler workflows instances describe [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. +- `ID` + - The ID of a Workflow instance. You can optionally provide `latest` to refer to the most recently created instance of a Workflow. + +```sh +# Passing `latest` instead of an explicit ID will describe the most recently queued instance +wrangler workflows instances describe my-workflow latest +``` +```sh output +Workflow Name: my-workflow +Instance Id: 51c73fc8-7fd5-47d9-bd82-9e301506ee72 +Version Id: cedc33a0-11fa-4c26-8a8e-7d28d381a291 +Status: βœ… Completed +Trigger: 🌎 API +Queued: 10/16/2024, 2:00:39 PM +Success: βœ… Yes +Start: 10/16/2024, 2:00:39 PM +End: 10/16/2024, 2:01:40 PM +Duration: 1 minute +# Remaining output truncated +``` + +### `instances terminate` + +Terminate (permanently stop) a Workflow instance. + +```sh +wrangler workflows instances terminate [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. +- `ID` + - The ID of a Workflow instance. + +{/* +### `instances pause` + +Pause (until resumed) a Workflow instance. + +```sh +wrangler workflows instances pause [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. +- `ID` + - The ID of a Workflow instance. + +### `instances resume` + +Resume a paused Workflow instance. + +```sh +wrangler workflows instances resume [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. +- `ID` + - The ID of a Workflow instance. + +*/} + +### `describe` + +```sh +wrangler workflows describe [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. + +### `trigger` + +Trigger (create) a Workflow instance. +```sh +wrangler workflows describe [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. +- `PARAMS` + - The parameters to pass to the Workflow as an event. Must be a JSON-encoded string. + + ```sh + # Pass optional params to the Workflow. + wrangler workflows instances trigger my-workflow '{"hello":"world"}' + ``` + +### `delete` + +Delete (unregister) a Workflow. + +```sh +wrangler workflows delete [OPTIONS] +``` + +- `WORKFLOW_NAME` + - The name of a registered Workflow. + ## `tail` Start a session to livestream logs from a deployed Worker. diff --git a/src/content/docs/workflows/build/events-and-parameters.mdx b/src/content/docs/workflows/build/events-and-parameters.mdx new file mode 100644 index 000000000000000..50b79bf839b44a5 --- /dev/null +++ b/src/content/docs/workflows/build/events-and-parameters.mdx @@ -0,0 +1,104 @@ +--- +title: Events and parameters +pcx_content_type: concept +sidebar: + order: 10 + +--- + +When a Workflow is triggered, it can receive an optional event. This event can include data that your Workflow can act on, including request details, user data fetched from your database (such as D1 or KV) or from a webhook, or messages from a Queue consumer. + +Events are a powerful part of a Workflow, as you often want a Workflow to act on data. Because a given Workflow instance executes durably, events are a useful way to provide a Workflow with data that should be immutable (not changing) and/or represents data the Workflow needs to operate on at that point in time. + +## Pass parameters to a Workflow + +You can pass parameters to a Workflow in two ways: + +* As an optional argument to the `create` method on a [Workflow binding](/workers/wrangler/commands/#trigger) when triggering a Workflow from a Worker. +* Via the `--params` flag when using the `wrangler` CLI to trigger a Workflow. + +You can pass any JSON-serializable object as a parameter. + +:::caution + +A `WorkflowEvent` and its associated `payload` property are effectively _immutable_: any changes to an event are not persisted across the steps of a Workflow. This includes both cases when a Workflow is progressing normally, and in cases where a Workflow has to be restarted due to a failure. + +Store state durably by returning it from your `step.do` callbacks. + +::: + +```ts +export default { + async fetch(req: Request, env: Env) { + let someEvent = { url: req.url, createdTimestamp: Date.now() } + // Trigger our Workflow + // Pass our event as the second parameter to the `create` method + // on our Workflow binding. + let instance = await env.MYWORKFLOW.create(await crypto.randomUUID(), someEvent); + + return Response.json({ + id: instance.id, + details: await instance.status(), + }); + + return Response.json({ result }); + }, +}; +``` + +To pass parameters via the `wrangler` command-line interface, pass a JSON string as the second parameter to the `workflows trigger` sub-command: + +```sh +npx wrangler@latest workflows trigger workflows-starter '{"some":"data"}' +``` +```sh output +πŸš€ Workflow instance "57c7913b-8e1d-4a78-a0dd-dce5a0b7aa30" has been queued successfully +``` + +## TypeScript and type parameters + +By default, the `WorkflowEvent` passed to the `run` method of your Workflow definition has a type that conforms to the following, with `payload` (your data) and `timestamp` properties: + +```ts +export type WorkflowEvent = { + // The data passed as the parameter when the Workflow instance was triggered + payload: T; + // The timestamp that the Workflow was triggered + timestamp: Date; +}; +``` + +You can optionally type these events by defining your own type and passing it as a [type parameter](https://www.typescriptlang.org/docs/handbook/2/generics.html#working-with-generic-type-variables) to the `WorkflowEvent`: + +```ts +// Define a type that conforms to the events your Workflow instance is +// instantiated with +interface YourEventType { + userEmail: string; + createdTimestamp: number; + metadata?: Record; +} +``` + +When you pass your `YourEventType` to `WorkflowEvent` as a type parameter, the `event.payload` property now has the type `YourEventType` throughout your workflow definition: + +```ts title="src/index.ts" +// Import the Workflow definition +import { WorkflowEntrypoint, WorkflowStep, WorkflowEvent} from 'cloudflare:workers'; + +export class MyWorkflow extends WorkflowEntrypoint { + // Pass your type as a type parameter to WorkflowEvent + // The 'payload' property will have the type of your parameter. + async run(event: WorkflowEvent, step: WorkflowStep) { + let state = step.do("my first step", async () => { + // Access your properties via event.payload + let userEmail = event.payload.userEmail + let createdTimestamp = event.payload.createdTimestamp + }) + + step.do("my second step", async () => { /* your code here */ ) + } +} +``` + +Note that type parameters do not _validate_ that the incoming event matches your type definition. Properties (fields) that do not exist or conform to the type you provided will be dropped. If you need to validate incoming events, we recommend a library such as [zod](https://zod.dev/) or your own validator logic. diff --git a/src/content/docs/workflows/build/index.mdx b/src/content/docs/workflows/build/index.mdx new file mode 100644 index 000000000000000..aa19963235b2fcd --- /dev/null +++ b/src/content/docs/workflows/build/index.mdx @@ -0,0 +1,13 @@ +--- +title: Build with Workflows +pcx_content_type: navigation +sidebar: + order: 2 + group: + hideIndex: true + +--- + +import { DirectoryListing } from "~/components" + + diff --git a/src/content/docs/workflows/build/rules-of-workflows.mdx b/src/content/docs/workflows/build/rules-of-workflows.mdx new file mode 100644 index 000000000000000..b89258b3ddeb5b5 --- /dev/null +++ b/src/content/docs/workflows/build/rules-of-workflows.mdx @@ -0,0 +1,249 @@ +--- +title: Rules of Workflows +pcx_content_type: concept +sidebar: + order: 10 +--- + +A Workflow contains one or more steps. Each step is a self-contained, individually retriable component of a Workflow. Steps may emit (optional) state that allows a Workflow to persist and continue from that step, even if a Workflow fails due to a network or infrastructure issue. + +This is a small guidebook on how to build more resilient and correct Workflows. + +### Ensure API/Binding calls are idempotent + +Because a step might be retried multiple times, your steps should (ideally) be idempotent. For context, idempotency is a logical property where the operation (in this case a step), +can be applied multiple times without changing the result beyond the initial application. + +As an example, let us assume you have a Workflow that charges your customers, and you really do not want to charge them twice by accident. Before charging them, you should +check if they were already charged: + +```ts +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + const customer_id = 123456; + // βœ… Good: Non-idempotent API/Binding calls are always done **after** checking if the operation is + // still needed. + await step.do( + `charge ${customer_id} for it's montly subscription`, + async () => { + // API call to check if customer was already charged + const subscription = await fetch( + `https://payment.processor/subscriptions/${customer_id}`, + ).then((res) => res.json()); + + // return early if the customer was already charged, this can happen if the destination service dies + // in the middle of the request but still commits it, or if the Workflows Engine restarts. + if (subscription.charged) { + return; + } + + // non-idempotent call, this operation can fail and retry but still commit in the payment + // processor - which means that, on retry, it would mischarge the customer again if the above checks + // were not in place. + return await fetch( + `https://payment.processor/subscriptions/${customer_id}`, + { + method: "POST", + body: JSON.stringify({ amount: 10.0 }), + }, + ); + }, + ); + } +} +``` + +:::note + +Guaranteeing idempotency might be optional in your specific use-case and implementation, but we recommend that you always try to guarantee it. + +::: + +### Make your steps granular + +Steps should be as self-contained as possible. This allows your own logic to be more durable in case of failures in third-party APIs, network errors, and so on. + +You can also think of it as a transaction, or a unit of work. + +- βœ… Minimize the number of API/binding calls per step (unless you need multiple calls to prove idempotency). + +```ts +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // βœ… Good: Unrelated API/Binding calls are self-contained, so that in case one of them fails + // it can retry them individually. It also has an extra advantage: you can control retry or + // timeout policies for each granular step - you might not to want to overload http.cat in + // case of it being down. + const httpCat = await step.do("get cutest cat from KV", async () => { + return await env.KV.get("cutest-http-cat"); + }); + + const image = await step.do("fetch cat image from http.cat", async () => { + return await fetch(`https://http.cat/${httpCat}`); + }); + } +} +``` + +Otherwise, your entire Workflow might not be as durable as you might think, and you may encounter some undefined behaviour. You can avoid them by following the rules below: + +- πŸ”΄ Do not encapsulate your entire logic in one single step. +- πŸ”΄ Do not call separate services in the same step (unless you need it to prove idempotency). +- πŸ”΄ Do not make too many service calls in the same step (unless you need it to prove idempotency). +- πŸ”΄ Do not do too much CPU-intensive work inside a single step - sometimes the engine may have to restart, and it will start over from the beginning of that step. + +```ts +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // πŸ”΄ Bad: you're calling two seperate services from within the same step. This might cause + // some extra calls to the first service in case the second one fails, and in some cases, makes + // the step non-idempotent altogether + const image = await step.do("get cutest cat from KV", async () => { + const httpCat = await env.KV.get("cutest-http-cat"); + return fetch(`https://http.cat/${httpCat}`); + }); + } +} +``` + +### Do not rely on state outside of a step + +Workflows may hibernate and lose all in-memory state. This will happen when engine detects that there is no pending work and can hibernate until it needs to wake-up (because of a sleep, retry, or event). + +This means that you should not store state outside of a step: + +```ts +function getRandomInt(min, max) { + const minCeiled = Math.ceil(min); + const maxFloored = Math.floor(max); + return Math.floor(Math.random() * (maxFloored - minCeiled) + minCeiled); // The maximum is exclusive and the minimum is inclusive +} + +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // πŸ”΄ Bad: `imageList` will be not persisted across engine's lifetimes. Which means that after hibernation, + // `imageList` will be empty again, even though the following two steps have already ran. + const imageList: string[] = []; + + await step.do("get first cutest cat from KV", async () => { + const httpCat = await env.KV.get("cutest-http-cat-1"); + + imageList.append(httpCat); + }); + + await step.do("get second cutest cat from KV", async () => { + const httpCat = await env.KV.get("cutest-http-cat-2"); + + imageList.append(httpCat); + }); + + // A long sleep can (and probably will) hibernate the engine which means that the first engine lifetime ends here + await step.sleep("πŸ’€πŸ’€πŸ’€πŸ’€", "3 hours"); + + // When this runs, it will be on the second engine lifetime - which means `imageList` will be empty. + await step.do( + "choose a random cat from the list and download it", + async () => { + const randomCat = imageList.at(getRandomInt(0, imageList.length)); + // this will fail since `randomCat` is undefined because `imageList` is empty + return await fetch(`https://http.cat/${randomCat}`); + }, + ); + } +} +``` + +Instead, you should build top-level state exclusively comprised of `step.do` returns: + +```ts +function getRandomInt(min, max) { + const minCeiled = Math.ceil(min); + const maxFloored = Math.floor(max); + return Math.floor(Math.random() * (maxFloored - minCeiled) + minCeiled); // The maximum is exclusive and the minimum is inclusive +} + +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // βœ… Good: imageList state is exclusively comprised of step returns - this means that in the event of + // multiple engine lifetimes, imageList will be built accordingly + const imageList: string[] = await Promise.all([ + step.do("get first cutest cat from KV", async () => { + return await env.KV.get("cutest-http-cat-1"); + }), + + step.do("get second cutest cat from KV", async () => { + return await env.KV.get("cutest-http-cat-2"); + }), + ]); + + // A long sleep can (and probably will) hibernate the engine which means that the first engine lifetime ends here + await step.sleep("πŸ’€πŸ’€πŸ’€πŸ’€", "3 hours"); + + // When this runs, it will be on the second engine lifetime - but this time, imageList will contain + // the two most cutest cats + await step.do( + "choose a random cat from the list and download it", + async () => { + const randomCat = imageList.at(getRandomInt(0, imageList.length)); + // this will eventually succeed since `randomCat` is defined + return await fetch(`https://http.cat/${randomCat}`); + }, + ); + } +} +``` + +### Do not mutate your incoming events + +The `event` passed to your Workflow's `run` method is immutable: changes you make to the event are not persisted across steps and/or Workflow restarts. + +```ts +interface MyEvent { + user: string; + data: string; +} + +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // πŸ”΄ Bad: Mutating the event + // This will not be persisted across steps and `event.data` will + // take on its original value. + await step.do("bad step that mutates the incoming event", async () => { + let userData = await env.KV.get(event.user) + event.data = userData + }) + + // βœ… Good: persist data by returning it as state from your step + // Use that state in subsequent steps + let userData = await step.do("good step that returns state", async () => { + return await env.KV.get(event.user) + }) + + let someOtherData await step.do("following step that uses that state", async () => { + // Access to userData here + // Will always be the same if this step is retried + }) +``` + +### Name steps deterministically + +Dynamically naming a step will prevent it from being cached, and cause the step to be re-run unnecessarily. Step names act as the "cache key" in your Workflow. + +```ts +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // πŸ”΄ Bad: Dynamically naming the step prevents it from being cached + // This will cause the step to be re-run if subsequent steps fail. + await step.do(`step #1 running at: ${Date.now}`, async () => { + let userData = await env.KV.get(event.user) + event.data = userData + }) + + // βœ… Good: give steps a deterministic name. + // Return dynamic values in your state, or log them instead. + let state = await step.do("fetch user data from KV", async () => { + let userData = await env.KV.get(event.user) + console.log(`fetched at ${Date.now}) + return userData + }) +``` diff --git a/src/content/docs/workflows/build/sleeping-and-retrying.mdx b/src/content/docs/workflows/build/sleeping-and-retrying.mdx new file mode 100644 index 000000000000000..4836e04fb4dd5e5 --- /dev/null +++ b/src/content/docs/workflows/build/sleeping-and-retrying.mdx @@ -0,0 +1,112 @@ +--- +title: Sleeping and retrying +pcx_content_type: concept +sidebar: + order: 4 + +--- + +This guide details how to sleep a Workflow and/or configure retries for a Workflow step. + +## Sleep a Workflow + +You can set a Workflow to sleep as an explicit step, which can be useful when you want a Workflow to wait, schedule work ahead, or pause until an input or other external state is ready. + +:::note + +A Workflow instance that is resuming from sleep will take priority over newly scheduled (queued) instances. This helps ensure that older Workflow instances can run to completion and are not blocked by newer instances. + +::: + +### Sleep for a relative period + +Use `step.sleep` to have a Workflow sleep for a relative period of time: + +```ts +await step.sleep("sleep for a bit", "1 hour") +``` + +The second argument to `step.sleep` accepts both `number` (seconds) or a human-readable format, such as "1 minute" or "26 hours". The accepted units for `step.sleep` when used this way are as follows: + +```ts +| "second" +| "minute" +| "hour" +| "day" +| "week" +| "month" +| "year" +``` + +### Sleep until a fixed date + +Use `step.sleepUntil` to have a Workflow sleep to a specific `Date`: this can be useful when you have a timestamp from another system or want to "schedule" work to occur at a specific time (e.g. Sunday, 9AM UTC). + +```ts +// sleepUntil accepts a Date object as its second argument +const workflowsLaunchDate = Date.parse("24 Oct 2024 13:00:00 UTC"); +await step.sleepUntil("sleep until X times out", workflowsLaunchDate) +``` + +You can also provide a Unix timestamp (seconds since the Unix epoch) directly to `sleepUntil`. + +## Retry steps + +Each call to `step.do` in a Workflow accepts an optional `StepConfig`, which allows you define the retry behaviour for that step. + +If you do not provide your own retry configuration, Workflows applies the following defaults: + +```ts +const defaultConfig: WorkflowStepConfig = { + retries: { + limit: 5, + delay: 10000, + backoff: 'exponential', + }, + timeout: '10 minutes', +}; +``` + +When providing your own `StepConfig`, you can configure: + +* The total number of attempts to make for a step +* The delay between attempts +* What backoff algorithm to apply between each attempt: any of `constant`, `linear`, or `exponential` +* When to timeout (in duration) before considering the step as failed (including during a retry attempt) + +For example, to limit a step to 10 retries and have it apply an exponential delay (starting at 10 seconds) between each attempt, you would pass the following configuration as an optional object to `step.do`: + +```ts +let someState = step.do("call an API", { + retries: { + limit: 10, // The total number of attempts + delay: "10 seconds", // Delay between each retry + backoff: "exponential" // Any of "constant" | "linear" | "exponential"; + }, + timeout: "30 minutes", +}, async () => { /* Step code goes here /* } +``` + +## Force a Workflow to fail + +You can also force a Workflow instance to fail and _not_ retry by throwing a `NonRetryableError` from within the step. + +This can be useful when you detect a terminal (permanent) error from an upstream system (such as an authentication failure) or other errors where retrying would not help. + +```ts +// Import the NonRetryableError definition +import { WorkflowEntrypoint, WorkflowStep, WorkflowEvent, NonRetryableError } from 'cloudflare:workers'; + +// In your step code: +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + await step.do("some step", async () => { + if !(event.data) { + throw NonRetryableError("event.data did not contain the expected payload") + } + }) + } +} +``` + +The Workflow instance itself will fail immediately, no further steps will be invoked, and the Workflow will not be retried. diff --git a/src/content/docs/workflows/build/trigger-workflows.mdx b/src/content/docs/workflows/build/trigger-workflows.mdx new file mode 100644 index 000000000000000..7b631d78a80f8a4 --- /dev/null +++ b/src/content/docs/workflows/build/trigger-workflows.mdx @@ -0,0 +1,174 @@ +--- +title: Trigger Workflows +pcx_content_type: concept +sidebar: + order: 3 + +--- + +You can trigger Workflows both programmatically and via the Workflows APIs, including: + +1. With [Workers](/workers) via HTTP requests in a `fetch` handler, or bindings from a `queue` or `scheduled` handler +2. Using the [Workflows REST API](/api/operations/wor-list-workflows) +2. Via the [wrangler CLI](/workers/wrangler/commands/#workflows) in your terminal + +## Workers API (Bindings) + +You can interact with Workflows programmatically from any Worker script by creating a binding to a Workflow. A Worker can bind to multiple Workflows, including Workflows defined in other Workers projects (scripts) within your account. + +You can interact with a Workflow: + +* Directly over HTTP via the [`fetch`](/workers/runtime-apis/handlers/fetch/) handler +* From a [Queue consumer](/queues/configuration/javascript-apis/#consumer) inside a `queue` handler +* From a [Cron Trigger](/workers/configuration/cron-triggers/) inside a `scheduled` handler +* Within a [Durable Object](/durable-objects/) + +:::note + +New to Workflows? Start with the [Workflows tutorial](/workflows/get-started/guide/) to deploy your first Workflow and familiarize yourself with Workflows concepts. + +::: + +To bind to a Workflow from your Workers code, you need to define a [binding](/workers/wrangler/configuration/) to a specific Workflow. For example, to bind to the Workflow defined in the [get started guide](/workflows/get-started/guide/), you would configure a `wrangler.toml` with the below: + +```toml title="wrangler.toml" +name = "workflows-tutorial" +main = "src/index.ts" +compatibility_date = "2024-10-15" + +[[workflows]] +# The name of the Workflow +name = "workflows-tutorial" +# The binding name, which must be a valid JavaScript variable name. This will +# be how you call (run) your Workflow from your other Workers handlers or +# scripts. +binding = "MY_WORKFLOW" + # script_name is required during for the beta. + # Must match the "name" of your Worker at the top of wrangler.toml +script_name = "workflows-tutorial" +# Must match the class defined in your code that extends the Workflow class +class_name = "MyWorkflow" +``` + +The `binding = "MY_WORKFLOW"` line defines the JavaScript variable that our Workflow methods are accessible on, including `create` (which triggers a new instance) or `get` (which returns the status of an existing instance). + +The following example shows how you can manage Workflows from within a Worker, including: + +* Retrieving the status of an existing Workflow instance by its ID +* Creating (triggering) a new Workflow instance +* Returning the status of a given instance ID + +```ts title="src/index.ts" +interface Env { + MY_WORKFLOW: Workflow; +} + +export default { + async fetch(req: Request, env: Env) { + // + const instanceId = new URL(req.url).searchParams.get("instanceId") + + // If an ?instanceId= query parameter is provided, fetch the status + // of an existing Workflow by its ID. + if (instanceId) { + let instance = await env.MY_WORKFLOW.get(id); + return Response.json({ + status: await instance.status(), + }); + } + + // Else, create a new instance of our Workflow, passing in any (optional) params + // and return the ID. + const newId = await crypto.randomUUID(); + let instance = await env.MY_WORKFLOW.create(newId, {}); + return Response.json({ + id: instance.id, + details: await instance.status(), + }); + + return Response.json({ result }); + }, +}; +``` + +### Inspect a Workflow's status + +You can inspect the status of any running Workflow instance by calling `status` against a specific instance ID. This allows you to programmatically inspect whether an instance is queued (waiting to be scheduled), actively running, paused, or errored. + +```ts +let instance = await env.MY_WORKFLOW.get("abc-123") +let status = await instance.status() // Returns an InstanceStatus +``` + +The possible values of status are as follows: + +```ts + status: + | "queued" // means that instance is waiting to be started (see concurrency limits) + | "running" + | "paused" + | "errored" + | "terminated" // user terminated the instance while it was running + | "complete" + | "waiting" // instance is hibernating and waiting for sleep or event to finish + | "waitingForPause" // instance is finishing the current work to pause + | "unknown"; + error?: string; + output?: object; +}; +``` +{/* +### Explicitly pause a Workflow + +You can explicitly pause a Workflow instance (and later resume it) by calling `pause` against a specific instance ID. + +```ts +let instance = await env.MY_WORKFLOW.get("abc-123") +await instance.pause() // Returns Promise +``` + +### Resume a Workflow + +You can resume a paused Workflow instance by calling `resume` against a specific instance ID. + +```ts +let instance = await env.MY_WORKFLOW.get("abc-123") +await instance.resume() // Returns Promise +``` + +Calling `resume` on an instance that is not currently paused will have no effect. +*/} + +### Stop a Workflow + +You can stop a Workflow instance by calling `abort` against a specific instance ID. + +```ts +let instance = await env.MY_WORKFLOW.get("abc-123") +await instance.abort() // Returns Promise +``` + +Once stopped, the Workflow instance *cannot* be resumed. + +### Restart a Workflow + +:::caution + +**Known issue**: Restarting a Workflow via the `restart()` method is not currently supported ad will throw an exception (error). + +::: + +```ts +let instance = await env.MY_WORKFLOW.get("abc-123") +await instance.restart() // Returns Promise +``` + +Restarting an instance will immediately cancel any in-progress steps, erase any intermediate state, and treat the Workflow as if it was run for the first time. + +## REST API (HTTP) + +Refer to the [Workflows REST API documentation](/api/operations/wor-create-new-workflow-instance). + +## Command line (CLI) + +Refer to the [CLI quick start](/workflows/get-started/cli-quick-start/) to learn more about how to manage and trigger Workflows via the command-line. diff --git a/src/content/docs/workflows/build/workers-api.mdx b/src/content/docs/workflows/build/workers-api.mdx new file mode 100644 index 000000000000000..b9c45e2dc15706f --- /dev/null +++ b/src/content/docs/workflows/build/workers-api.mdx @@ -0,0 +1,274 @@ +--- +title: Workers API +pcx_content_type: concept +sidebar: + order: 2 + +--- + +import { MetaInfo, Type } from "~/components"; + +This guide details the Workflows API within Cloudflare Workers, including methods, types, and usage examples. + +## WorkflowEntrypoint + +The `WorkflowEntrypoint` class is the core element of a Workflow definition. A Workflow must extend this class and define a `run` method with at least one `step` call to be considered a valid Workflow. + +```ts +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // Steps here + } +}; +``` + +* run(event: WorkflowEvent<T>, step: WorkflowStep): Promise<T> + + * `event` - the event passed to the Workflow, including an optional `payload` containing data (parameters) + * `step` - the `WorkflowStep` type that provides the step methods for your Workflow + +The `WorkflowEvent` type accepts an optional [type parameter](https://www.typescriptlang.org/docs/handbook/2/generics.html#working-with-generic-type-variables) that allows you to provide a type for the `payload` property within the `WorkflowEvent`. + +Refer to the [events and parameters](/workflows/build/events-and-parameters/) documentation for how to handle events within yur Workflow code. + +## WorkflowEvent + +```ts +export type WorkflowEvent = { + payload: Readonly; + timestamp: Date; +}; +``` + +* The `WorkflowEvent` is the first argument to a Workflow's `run` method, and includes an optional `payload` parameter and a `timestamp` property. + + * `payload` - a default type of `any` or type `T` if a type parameter is provided. + * `timestamp` - a `Date` object set to the time the Workflow instance was created (triggered). + +Refer to the [events and parameters](/workflows/build/events-and-parameters/) documentation for how to handle events within yur Workflow code. + +## WorkflowStep + +### step + +* step.do(name: string, callback: (): RpcSerializable): Promise<T> +* step.do(name: string, config?: WorkflowStepConfig, callback: (): RpcSerializable): Promise<T> + + * `name` - the name of the step. + * `config` (optional) - an optional `WorkflowStepConfig` for configuring [step specific retry behaviour](/workflows/build/sleeping-and-retrying/). + * `callback` - an asynchronous function that optionally returns serializable state for the Workflow to persist. + +* step.sleep(name: string, duration: WorkflowDuration): Promise<void> + + * `name` - the name of the step. + * `duration` - the duration to sleep until, in either seconds or as a `WorkflowDuration` compatible string. + * Refer to the [documentation on sleeping and retrying](/workflows/build/sleeping-and-retrying/) to learn more about how how Workflows are retried. + +* step.sleepUntil(name: string, timestamp: Date | number): Promise<void> + + * `name` - the name of the step. + * `timestamp` - a JavaScript `Date` object or seconds from the Unix epoch to sleep the Workflow instance until. + +## WorkflowStepConfig + +```ts +export type WorkflowStepConfig = { + retries?: { + limit: number; + delay: string | number; + backoff?: WorkflowBackoff; + }; + timeout?: string | number; +}; +``` + +* A `WorkflowStepConfig` is an optional argument to the `do` method of a `WorkflowStep` and defines properties that allow you to configure the retry behaviour of that step. + +Refer to the [documentation on sleeping and retrying](/workflows/build/sleeping-and-retrying/) to learn more about how how Workflows are retried. + +## NonRetryableError + +* throw new NonRetryableError(message: , name ): + + * Throws an error that forces the current Workflow instance to fail and not be retried. + * Refer to the [documentation on sleeping and retrying](/workflows/build/sleeping-and-retrying/) to learn more about how how Workflows are retried. + +## Call Workflows from Workers + +:::note[Workflows beta] + +Workflows currently requires you to bind to a Workflow via `wrangler.toml` and does not yet support bindings via the Workers dashboard. + +::: + +Workflows exposes an API directly to your Workers scripts via the [bindings](/workers/runtime-apis/bindings/#what-is-a-binding) concept. Bindings allow you to securely call a Workflow without having to manage API keys or clients. + +You can bind to a Workflow by defining a `[[workflows]]` binding within your `wrangler.toml` configuration. + +For example, to bind to a Workflow called `workflows-starter` and to make it available on the `MY_WORKFLOW` variable to your Worker script, you would configure the following fields within the `[[workflows]]` binding definition: + +```toml title="wrangler.toml" +#:schema node_modules/wrangler/config-schema.json +name = "workflows-starter" +main = "src/index.ts" +compatibility_date = "2024-10-16" + +[[workflows]] +# name of your workflow +name = "workflows-starter" +# binding name env.MYWORKFLOW +binding = "MY_WORKFLOW" +# this is class that extends the Workflow class in src/index.ts +class_name = "MyWorkflow" +# script_name is required during for the beta. +# Must match the "name" of your Worker at the top of wrangler.toml +script_name = "workflows-starter" +``` + +## Workflow + +:::note + +Ensure you have `@cloudflare/workers-types` version `4.20241022.0` or later installed when binding to Workflows from within a Workers project. + +::: + +The `Workflow` type provides methods that allow you to create, inspect the status, and manage running Workflow instances from within a Worker script. + +```ts +interface Env { + // The 'MY_WORKFLOW' variable should match the "binding" value set in wrangler.toml + MY_WORKFLOW: Workflow; +} +``` + +The `Workflow` type exports the following methods: + +### create + +Create (trigger) a new instance of the given Workflow. + +* create(options?: WorkflowInstanceCreateOptions): Promise<WorkflowInstance> + + * `options` - optional properties to pass when creating an instance. + +An ID is automatically generated, but a user-provided ID can be specified. This can be useful when mapping Workflows to users, merchants or other identifiers in your system. + +```ts +// Create a new Workflow instance with your own ID: +let instance = await env.MY_WORKFLOW.create({ id: myIdDefinedFromOtherSystem }) +return Response.json({ + id: instance.id, + details: await instance.status(), +}); +``` + +Returns a `WorkflowInstance`. + +### get + +Get a specific Workflow instance by ID. + +* get(id: string): Promise<WorkflowInstance> + + * `id` - the ID of the Workflow instance. + +Returns a `WorkflowInstance`. + +## WorkflowInstanceCreateOptions + +Optional properties to pass when creating an instance. + +```ts +interface WorkflowInstanceCreateOptions { + /** + * An id for your Workflow instance. Must be unique within the Workflow. + */ + id?: string; + /** + * The event payload the Workflow instance is triggered with + */ + params?: unknown; +} +``` + +## WorkflowInstance + +Represents a specific instance of a Workflow, and provides methods to manage the instance. + +```ts +declare abstract class WorkflowInstance { + public id: string; + /** + * Pause the instance. + */ + public pause(): Promise; + /** + * Resume the instance. If it is already running, an error will be thrown. + */ + public resume(): Promise; + /** + * Terminate the instance. If it is errored, terminated or complete, an error will be thrown. + */ + public terminate(): Promise; + /** + * Restart the instance. + */ + public restart(): Promise; + /** + * Returns the current status of the instance. + */ + public status(): Promise; +} +``` + +### id + +Return the id of a Workflow. + +* id: string + +### status + +Return the status of a running Workflow instance. + +* status(): Promise<void> + +### pause + +Pause a running Workflow instance. + +* pause(): Promise<void> + +### restart + +Restart a Workflow instance. + +* restart(): Promise<void> + +### terminate + +Terminate a Workflow instance. + +* terminate(): Promise<void> + +### InstanceStatus + +Details the status of a Workflow instance. + +```ts +type InstanceStatus = { + status: + | "queued" // means that instance is waiting to be started (see concurrency limits) + | "running" + | "paused" + | "errored" + | "terminated" // user terminated the instance while it was running + | "complete" + | "waiting" // instance is hibernating and waiting for sleep or event to finish + | "waitingForPause" // instance is finishing the current work to pause + | "unknown"; + error?: string; + output?: object; +}; +``` diff --git a/src/content/docs/workflows/examples/automate-lifecycle-emails.mdx b/src/content/docs/workflows/examples/automate-lifecycle-emails.mdx new file mode 100644 index 000000000000000..f6457998b11e626 --- /dev/null +++ b/src/content/docs/workflows/examples/automate-lifecycle-emails.mdx @@ -0,0 +1,15 @@ +--- +type: example +summary: Automate lifecycle emails +tags: + - Workflows + - Email +pcx_content_type: configuration +title: Automate lifecycle emails +sidebar: + order: 3 +description: Automate lifecycle emails using Workflows and Resend + +--- + +import { TabItem, Tabs } from "~/components" diff --git a/src/content/docs/workflows/examples/index.mdx b/src/content/docs/workflows/examples/index.mdx new file mode 100644 index 000000000000000..c305f7917ef78ce --- /dev/null +++ b/src/content/docs/workflows/examples/index.mdx @@ -0,0 +1,17 @@ +--- +type: overview +hideChildren: false +pcx_content_type: navigation +title: Examples +sidebar: + order: 6 + group: + hideIndex: true + +--- + +import { GlossaryTooltip, ListExamples } from "~/components" + +Explore the following examples for D1. + + diff --git a/src/content/docs/workflows/examples/post-process-r2.mdx b/src/content/docs/workflows/examples/post-process-r2.mdx new file mode 100644 index 000000000000000..8e79f7b54f477cf --- /dev/null +++ b/src/content/docs/workflows/examples/post-process-r2.mdx @@ -0,0 +1,15 @@ +--- +type: example +summary: Post-process files from R2 +tags: + - Workflows + - R2 +pcx_content_type: configuration +title: Post-process files from R2 +sidebar: + order: 3 +description: Post-process files from R2 object storage using Workflows + +--- + +import { TabItem, Tabs } from "~/components" diff --git a/src/content/docs/workflows/get-started/cli-quick-start.mdx b/src/content/docs/workflows/get-started/cli-quick-start.mdx new file mode 100644 index 000000000000000..bad746ae9faae60 --- /dev/null +++ b/src/content/docs/workflows/get-started/cli-quick-start.mdx @@ -0,0 +1,271 @@ +--- +title: CLI quick start +pcx_content_type: get-started +updated: 2024-10-23 +sidebar: + order: 3 + +--- + +import { Render, PackageManagers } from "~/components" + +:::note + +Workflows is in **public beta**, and any developer with a [free or paid Workers plan](/workers/platform/pricing/#workers) can start using Workflows immediately. + +To learn more about Workflows and how it works, read [the beta announcement blog](https://blog.cloudflare.com/building-workflows-durable-execution-on-workers). + +::: + +Workflows allow you to build durable, multi-step applications using the Workers platform. A Workflow can automatically retry, persist state, run for hours or days, and coordinate between third-party APIs. + +You can build Workflows to post-process file uploads to [R2 object storage](/r2/), automate generation of [Workers AI](/workers-ai/) embeddings into a [Vectorize](/vectorize/) vector database, or to trigger user lifecycle emails using your favorite email API. + +## Prerequisites + +:::caution + +This guide is for users who are already familiar with Cloudflare Workers the [durable execution](/workflows/reference/glossary/) programming model it enables. + +If you are new to either, we recommend the [introduction to Workflows](/workflows/get-started/guide/) guide, which walks you through how a Workflow is defined, how to persist state, and how to deploy and run your first Workflow. + +::: + + +## 1. Create a Workflow + +Workflows are defined as part of a Worker script. + +To create a Workflow, use the `create cloudflare` (C3) CLI tool, specifying the Workflows starter template: + +```sh +npm create cloudflare@latest workflows-starter -- --template "cloudflare/workflows-starter" +``` + +This will create a new folder called `workflows-tutorial`, which contains two files: + +* `src/index.ts` - this is where your Worker script, including your Workflows definition, is defined. +* `wrangler.toml` - the configuration for your Workers project and your Workflow. + +Open the `src/index.ts` file in your text editor. This file contains the following code, which is the most basic instance of a Workflow definition: + +```ts title="src/index.ts" +import { WorkflowEntrypoint, WorkflowStep, WorkflowEvent } from 'cloudflare:workers'; + +type Env = { + // Add your bindings here, e.g. Workers KV, D1, Workers AI, etc. + MY_WORKFLOW: Workflow; +}; + +// User-defined params passed to your workflow +type Params = { + email: string; + metadata: Record; +}; + +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // Can access bindings on `this.env` + // Can access params on `event.params` + + const files = await step.do('my first step', async () => { + // Fetch a list of files from $SOME_SERVICE + return { + inputParams: event, + files: [ + 'doc_7392_rev3.pdf', + 'report_x29_final.pdf', + 'memo_2024_05_12.pdf', + 'file_089_update.pdf', + 'proj_alpha_v2.pdf', + 'data_analysis_q2.pdf', + 'notes_meeting_52.pdf', + 'summary_fy24_draft.pdf', + ], + }; + }); + + const apiResponse = await step.do('some other step', async () => { + let resp = await fetch('https://api.cloudflare.com/client/v4/ips'); + return await resp.json(); + }); + + await step.sleep('wait on something', '1 minute'); + + await step.do( + 'make a call to write that could maybe, just might, fail', + // Define a retry strategy + { + retries: { + limit: 5, + delay: '5 second', + backoff: 'exponential', + }, + timeout: '15 minutes', + }, + async () => { + // Do stuff here, with access to the state from our previous steps + if (Math.random() > 0.5) { + throw new Error('API call to $STORAGE_SYSTEM failed'); + } + }, + ); + } +} + +export default { + async fetch(req: Request, env: Env): Promise { + let id = new URL(req.url).searchParams.get('instanceId'); + + // Get the status of an existing instance, if provided + if (id) { + let instance = await env.MY_WORKFLOW.get(id); + return Response.json({ + status: await instance.status(), + }); + } + + // Spawn a new instance and return the ID and status + let instance = await env.MY_WORKFLOW.create(); + return Response.json({ + id: instance.id, + details: await instance.status(), + }); + }, +}; +``` + +Specifically, the code above: + +1. Extends the Workflows base class (`WorkflowsEntrypoint`) and defines a `run` method for our Workflow. +2. Passes in our `Params` type as a [type parameter](/workflows/build/events-and-parameters/) so that events that trigger our Workflow are typed. +3. Defines several steps that return state. +4. Defines a custom retry configuration for a step. +5. Binds to the Workflow from a Worker's `fetch` handler so that we can create (trigger) instances of our Workflow via a HTTP call. + +You can edit this Workflow by adding (or removing) additional `step` calls, changing the retry configuration, and/or making your own API calls. This Workflow template is designed to illustrate some of Workflows APIs. + +## 2. Deploy a Workflow + +Workflows are deployed via [`wrangler`](/workers/wrangler/install-and-update/), which is installed when you first ran `npm create cloudflare` above. Workflows are Worker scripts, and are deployed the same way: + +```sh +npx wrangler@latest deploy +``` + +## 3. Run a Workflow + +You can run a Workflow via the `wrangler` CLI, via a Worker binding, or via the Workflows [REST API](/api/operations/wor-list-workflows). + +### `wrangler` CLI + +```sh +# Trigger a Workflow from the CLI, and pass (optional) parameters as an event to the Workflow. +npx wrangler@latest workflows trigger workflows-tutorial --params={"hello":"world"} +``` + +Refer to the [events and parameters documentation](/workflows/build/events-and-parameters/) to understand how events are passed to Workflows. + +### Worker binding + +You can [bind to a Workflow](/workers/runtime-apis/bindings/#what-is-a-binding) from any handler in a Workers script, allowing you to programatically trigger and pass parameters to a Workflow instance from your own application code. + +To bind a Workflow to a Worker, you need to define a `[[workflows]]` binding in your `wrangler.toml` configuration: + +```toml title="wrangler.toml" +[[workflows]] +# name of your workflow +name = "workflows-starter" +# binding name env.MYWORKFLOW +binding = "MY_WORKFLOW" +# this is class that extends the Workflow class in src/index.ts +class_name = "MyWorkflow" +# script_name is required during for the beta. +# Must match the "name" of your Worker at the top of wrangler.toml +script_name = "workflows-starter" +``` + +You can then invoke the methods on this binding directly from your Worker script's `env` parameter. The `Workflow` type has methods for: + +* `create()` - creating (triggering) a new instance of the Workflow, returning the ID. +* `get()`- retrieve a Workflow instance by its ID. +* `status()` - get the current status of a unique Workflow instance. + +For example, the following Worker will fetch the status of an existing Workflow instance by ID (if supplied), else it will create a new Workflow instance and return its ID: + +```ts title="src/index.ts" +// Import the Workflow definition +import { WorkflowEntrypoint, WorkflowStep, WorkflowEvent} from 'cloudflare:workers'; + +interface Env { + // Matches the binding definition in your wrangler.toml + MY_WORKFLOW: Workflow; +} + +export default { + async fetch(req: Request, env: Env): Promise { + let id = new URL(req.url).searchParams.get('instanceId'); + + // Get the status of an existing instance, if provided + if (id) { + let instance = await env.MY_WORKFLOW.get(id); + return Response.json({ + status: await instance.status(), + }); + } + + // Spawn a new instance and return the ID and status + let instance = await env.MY_WORKFLOW.create(); + return Response.json({ + id: instance.id, + details: await instance.status(), + }); + }, +}; +``` + +Refer to the [triggering Workflows](/workflows/build/trigger-workflows/) documentation for how to trigger a Workflow from other Workers' handler functions. + +## 4. Manage Workflows + +:::note + +The `wrangler workflows` command requires Wrangler version `3.83.0` or greater. Use `npx wrangler@latest` to always use the latest Wrangler version when invoking commands. + +::: + +The `wrangler workflows` command group has several sub-commands for managing and inspecting Workflows and their instances: + +* List Workflows: `wrangler workflows list` +* Inspect the instances of a Workflow: `wrangler workflows instances list YOUR_WORKFLOW_NAME` +* View the state of a running Workflow instance by its ID: `wrangler workflows instances describe YOUR_WORKFLOW_NAME WORKFLOW_ID` + +You can also view the state of the latest instance of a Workflow by using the `latest` keyword instead of an ID: + +```sh +npx wrangler@latest workflows instances describe workflows-starter latest +# Or by ID: +# npx wrangler@latest workflows instances describe workflows-starter 12dc179f-9f77-4a37-b973-709dca4189ba +``` + +The output of `instances describe` shows: + +* The status (success, failure, running) of each step +* Any state emitted by the step +* Any `sleep` state, including when the Workflow will wake up +* Retries associated with each step +* Errors, including exception messages + +:::note + +You do not have to wait for a Workflow instance to finish executing to inspect its current status. The `wrangler workflows instances describe` sub-command will show the status of an in-progress instance, including any persisted state, if it is sleeping, and any errors or retries. This can be especially useful when debugging a Workflow during development. + +::: + +## Next steps + +* Learn more about [how events are passed to a Workflow](/workflows/build/events-and-parameters/). +* Binding to and triggering Workflow instances using the [Workers API](/workflows/build/workers-api/). +* The [Rules of Workflows](/workflows/build/rules-of-workflows/) and best practices for building applications using Workflows. + +If you have any feature requests or notice any bugs, share your feedback directly with the Cloudflare team by joining the [Cloudflare Developers community on Discord](https://discord.cloudflare.com). diff --git a/src/content/docs/workflows/get-started/guide.mdx b/src/content/docs/workflows/get-started/guide.mdx new file mode 100644 index 000000000000000..85f90a645813da3 --- /dev/null +++ b/src/content/docs/workflows/get-started/guide.mdx @@ -0,0 +1,483 @@ +--- +title: Guide +pcx_content_type: get-started +updated: 2024-10-23 +sidebar: + order: 1 + +--- + +import { Render, PackageManagers } from "~/components" + +:::note + +Workflows is in **public beta**, and any developer with a [free or paid Workers plan](/workers/platform/pricing/#workers) can start using Workflows immediately. + +To learn more about Workflows and how it works, read [the beta announcement blog](https://blog.cloudflare.com/building-workflows-durable-execution-on-workers). + +::: + +Workflows allow you to build durable, multi-step applications using the Workers platform. A Workflow can automatically retry, persist state, run for hours or days, and coordinate between third-party APIs. + +You can build Workflows to post-process file uploads to [R2 object storage](/r2/), automate generation of [Workers AI](/workers-ai/) embeddings into a [Vectorize](/vectorize/) vector database, or to trigger user lifecycle emails using your favorite email API. + +This guide will instruct you through: + +* Defining your first Workflow and publishing it +* Deploying the Workflow to your Cloudflare account +* Running (triggering) your Workflow and observing its output + +At the end of this guide, you should be able to author, deploy and debug your own Workflows applications. + +## Prerequisites + + + +## 1. Define your Workflow + +To create your first Workflow, use the `create cloudflare` (C3) CLI tool, specifying the Workflows starter template: + +```sh +npm create cloudflare@latest workflows-starter -- --template "cloudflare/workflows-starter" +``` + +This will create a new folder called `workflows-starter`. + +Open the `src/index.ts` file in your text editor. This file contains the following code, which is the most basic instance of a Workflow definition: + +```ts title="src/index.ts" +// Import the Workflow definition +import { WorkflowEntrypoint, WorkflowEvent, WorkflowStep } from "cloudflare:workers" + +// Create your own class that implements a Workflow +export class MyWorkflow implements WorkflowEntrypoint { + // Define a run() method + async run(event: WorkflowEvent, step: WorkflowStep) { + // Define one or more steps that optionally return state. + const state = await step.do('my first step', async () => { + return {} + }); + + await step.sleep('wait on something', '1 minute'); + + await step.do( + 'make a call to write that could maybe, just might, fail', + async () => { + // Do stuff here, with access to 'state' from the previous step + if (Math.random() > 0.5) { + throw new Error('API call to $STORAGE_SYSTEM failed'); + } + } + ); + } +} +``` + +A Workflow definition: + +1. Defines a `run` method that contains the primary logic for your workflow. +2. Has at least one or more calls to `step.do` that encapsulates the logic of your Workflow. +3. Allows steps to return (optional) state, allowing a Workflow to continue execution even if subsequent steps fail, without having to re-run all previous steps. + +A single Worker application can contain multiple Workflow definitions, as long as each Workflow has a unique class name. This can be useful for code re-use or to define Workflows which are related to each other conceptually. + +Each Workflow is otherwise entirely independent: a Worker that defines multiple Workflows is no different from a set of Workers that define one Workflow each. + +## 2. Create your Workflows steps + +Each `step` in a Workflow is an independently retriable function. + +A `step` is what makes a Workflow powerful, as you can encapsulate errors and persist state as your Workflow progresses from step to step, avoiding your application from having to start from scratch on failure and ultimately build more reliable applications. + +* A step can execute code (`step.do`) or sleep a Workflow (`step.sleep`). +* If a step fails (throws an exception), it will be automatically be retried based on your retry logic. +* If a step succeeds, any state it returns will be persisted within the Workflow. + +At its most basic, a step looks like this: + +```ts title="src/index.ts" +// Import the Workflow definition +import { WorkflowEntrypoint, WorkflowEvent, WorkflowStep } from "cloudflare:workers" + +// Create your own class that implements a Workflow +export class MyWorkflow implements WorkflowEntrypoint { + // Define a run() method + async run(event: WorkflowEvent, step: WorkflowStep) { + // Define one or more steps that optionally return state. + let state = step.do("my first step", async () => { + + }) + + step.do("my second step", async () => { + + }) + } +} +``` + +Each call to `step.do` accepts three arguments: + +1. (Required) A step name, which identifies the step in logs and telemetry +2. (Required) A callback function that contains the code to run for your step, and any state you want the Workflow to persist +3. (Optional) A `StepConfig` that defines the retry configuration (max retries, delay, and backoff algorithm) for the step + +When trying to decide whether to break code up into more than one step, a good rule of thumb is to ask "do I want _all_ of this code to run again if just one part of it fails?". In many cases, you do _not_ want to repeatedly call an API if the following data processing stage fails, or if you get an error when attempting to send a completion or welcome email. + +For example, each of the below tasks is ideally encapsulated in its own step, so that any failure β€” such as a file not existing, a third-party API being down or rate limited β€” does not cause your entire program to fail. + +* Reading or writing files from R2 +* Running an AI task using [Workers AI](/workers-ai/) +* Querying a D1 database or a database via [Hyperdrive](/hyperdrive/) +* Calling a third-party API + +If a subsequent step fails, your Workflow can retry from that step, using any state returned from a previous step. This can also help you avoid unnecessarily querying a database or calling an paid API repeatedly for data you have already fetched. + +:::note + +The term "Durable Execution" is widely used to describe this programming model. + +"Durable" describes the ability of the program (application) to implicitly persist state without you having to manually write to an external store or serialize program state. + +::: + +## 3. Configure your Workflow + +Before you can deploy a Workflow, you need to configure it. + +Open the `wrangler.toml` file at the root of your `workflows-starter` folder, which contains the following `[[workflows]]` configuration: + +```toml title="wrangler.toml" +#:schema node_modules/wrangler/config-schema.json +name = "workflows-starter" +main = "src/index.ts" +compatibility_date = "2024-10-23" + +[[workflows]] +# name of your workflow +name = "workflows-starter" +# binding name env.MYWORKFLOW +binding = "MY_WORKFLOW" +# this is class that extends the Workflow class in src/index.ts +class_name = "MyWorkflow" +# script_name is required during for the beta. +# Must match the "name" of your Worker at the top of wrangler.toml +script_name = "workflows-starter" +``` + +:::note + +If you have changed the name of the Workflow in your Wrangler commands, the JavaScript class name, or the name of the project you created, ensure that you update the values above to match the changes. + +::: + +This configuration tells the Workers platform which JavaScript class represents your Workflow, and sets a `binding` name that allows you to run the Workflow from other handlers or to call into Workflows from other Workers scripts. + +## 4. Bind to your Workflow + +We have a very basic Workflow definition, but now need to provide a way to call it from within our code. A Workflow can be triggered by: + +1. External HTTP requests via a `fetch()` handler +2. Messages from a [Queue](/queues/) +3. A schedule via [Cron Trigger](/workers/configuration/cron-triggers/) +4. Via the [Workflows REST API](/api/operations/wor-list-workflows) or [wrangler CLI](/workers/wrangler/commands/#workflows) + +Return to the `src/index.ts` file we created in the previous step and add a `fetch` handler that _binds_ to our Workflow. This binding allows us to create new Workflow instances, fetch the status of an existing Workflow, pause and/or terminate a Workflow. + +```ts title="src/index.ts" +// This can be in the same file as our Workflow definition + +export default { + async fetch(req: Request, env: Env): Promise { + let id = new URL(req.url).searchParams.get('instanceId'); + + // Get the status of an existing instance, if provided + if (id) { + let instance = await env.MY_WORKFLOW.get(id); + return Response.json({ + status: await instance.status(), + }); + } + + // Spawn a new instance and return the ID and status + let instance = await env.MY_WORKFLOW.create(); + return Response.json({ + id: instance.id, + details: await instance.status(), + }); + }, +}; +; +``` + +The code here exposes a HTTP endpoint that generates a random ID and runs the Workflow, returning the ID and the Workflow status. It also accepts an optional `instanceId` query parameter that retrieves the status of a Workflow instance by its ID. + +:::note + +In a production application, you might choose to put authentication in front of your endpoint so that only authorized users can run a Workflow. Alternatively, you could pass messages to a Workflow [from a Queue consumer](/queues/reference/how-queues-works/#consumers) in order to allow for long-running tasks. + +::: + +### Review your Workflow code + +:::note + +This is the full contents of the `src/index.ts` file pulled down when you used the `cloudflare/workflows-starter` template at the beginning of this guide. + +::: + +Before you deploy, you can review the full Workflows code and the `fetch` handler that will allow you to trigger your Workflow over HTTP: + +```ts title="src/index.ts" +import { WorkflowEntrypoint, WorkflowStep, WorkflowEvent } from 'cloudflare:workers'; + +type Env = { + // Add your bindings here, e.g. Workers KV, D1, Workers AI, etc. + MY_WORKFLOW: Workflow; +}; + +// User-defined params passed to your workflow +type Params = { + email: string; + metadata: Record; +}; + +export class MyWorkflow extends WorkflowEntrypoint { + async run(event: WorkflowEvent, step: WorkflowStep) { + // Can access bindings on `this.env` + // Can access params on `event.params` + + const files = await step.do('my first step', async () => { + // Fetch a list of files from $SOME_SERVICE + return { + inputParams: event, + files: [ + 'doc_7392_rev3.pdf', + 'report_x29_final.pdf', + 'memo_2024_05_12.pdf', + 'file_089_update.pdf', + 'proj_alpha_v2.pdf', + 'data_analysis_q2.pdf', + 'notes_meeting_52.pdf', + 'summary_fy24_draft.pdf', + ], + }; + }); + + const apiResponse = await step.do('some other step', async () => { + let resp = await fetch('https://api.cloudflare.com/client/v4/ips'); + return await resp.json(); + }); + + await step.sleep('wait on something', '1 minute'); + + await step.do( + 'make a call to write that could maybe, just might, fail', + // Define a retry strategy + { + retries: { + limit: 5, + delay: '5 second', + backoff: 'exponential', + }, + timeout: '15 minutes', + }, + async () => { + // Do stuff here, with access to the state from our previous steps + if (Math.random() > 0.5) { + throw new Error('API call to $STORAGE_SYSTEM failed'); + } + }, + ); + } +} + +export default { + async fetch(req: Request, env: Env): Promise { + let id = new URL(req.url).searchParams.get('instanceId'); + + // Get the status of an existing instance, if provided + if (id) { + let instance = await env.MY_WORKFLOW.get(id); + return Response.json({ + status: await instance.status(), + }); + } + + // Spawn a new instance and return the ID and status + let instance = await env.MY_WORKFLOW.create(); + return Response.json({ + id: instance.id, + details: await instance.status(), + }); + }, +}; +``` + +## 5. Deploy your Workflow + +Deploying a Workflow is identical to deploying a Worker. + +```sh +npx wrangler deploy +``` +```sh output +# Note the "Workflows" binding mentioned here, showing that +# wrangler has detected your Workflow +Your worker has access to the following bindings: +- Workflows: + - MY_WORKFLOW: MyWorkflow (defined in workflows-starter) +Uploaded workflows-starter (2.53 sec) +Deployed workflows-starter triggers (1.12 sec) + https://workflows-starter.silverlock.workers.dev + workflow: workflows-starter +``` + +A Worker with a valid Workflow definition will be automatically registered by Workflows. You can list your current Workflows using Wrangler: + +```sh +npx wrangler workflows list +``` +```sh output +Showing last 1 workflow: +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Name β”‚ Script name β”‚ Class name β”‚ Created β”‚ Modified β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ workflows-starter β”‚ workflows-starter β”‚ MyWorkflow β”‚ 10/23/2024, 11:33:58 AM β”‚ 10/23/2024, 11:33:58 AM β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## 6. Run and observe your Workflow + +With your Workflow deployed, you can now run it. + +1. A Workflow can run in parallel: each unique invocation of a Workflow is an _instance_ of that Workflow. +2. An instance will run to completion (success or failure). +3. Deploying newer versions of a Workflow will cause all instances after that point to run the newest Workflow code. + +:::note + +Because Workflows can be long running, it is possible to have running instances that represent different versions of your Workflow code over time. + +::: + +To trigger our Workflow, we will use the `wrangler` CLI and pass in an optional `--payload`. The `payload` will be passed to your Workflow's `run` method handler as an `Event`. + +```sh +npx wrangler workflows trigger workflows-starter '{"hello":"world"}' +``` +```sh output +# Workflow instance "12dc179f-9f77-4a37-b973-709dca4189ba" has been queued successfully +``` + +To inspect the current status of the Workflow instance we just triggered, we can either reference it by ID or by using the keyword `latest`: + +```sh +npx wrangler@latest workflows instances describe workflows-starter latest +# Or by ID: +# npx wrangler@latest workflows instances describe workflows-starter 12dc179f-9f77-4a37-b973-709dca4189ba +``` +```sh output +Workflow Name: workflows-starter +Instance Id: f72c1648-dfa3-45ea-be66-b43d11d216f8 +Version Id: cedc33a0-11fa-4c26-8a8e-7d28d381a291 +Status: βœ… Completed +Trigger: 🌎 API +Queued: 10/15/2024, 1:55:31 PM +Success: βœ… Yes +Start: 10/15/2024, 1:55:31 PM +End: 10/15/2024, 1:56:32 PM +Duration: 1 minute +Last Successful Step: make a call to write that could maybe, just might, fail-1 +Steps: + + Name: my first step-1 + Type: 🎯 Step + Start: 10/15/2024, 1:55:31 PM + End: 10/15/2024, 1:55:31 PM + Duration: 0 seconds + Success: βœ… Yes + Output: "{\"inputParams\":[{\"timestamp\":\"2024-10-15T13:55:29.363Z\",\"payload\":{\"hello\":\"world\"}}],\"files\":[\"doc_7392_rev3.pdf\",\"report_x29_final.pdf\",\"memo_2024_05_12.pdf\",\"file_089_update.pdf\",\"proj_alpha_v2.pdf\",\"data_analysis_q2.pdf\",\"notes_meeting_52.pdf\",\"summary_fy24_draft.pdf\",\"plan_2025_outline.pdf\"]}" +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Start β”‚ End β”‚ Duration β”‚ State β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ 10/15/2024, 1:55:31 PM β”‚ 10/15/2024, 1:55:31 PM β”‚ 0 seconds β”‚ βœ… Success β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + + Name: some other step-1 + Type: 🎯 Step + Start: 10/15/2024, 1:55:31 PM + End: 10/15/2024, 1:55:31 PM + Duration: 0 seconds + Success: βœ… Yes + Output: "{\"result\":{\"ipv4_cidrs\":[\"173.245.48.0/20\",\"103.21.244.0/22\",\"103.22.200.0/22\",\"103.31.4.0/22\",\"141.101.64.0/18\",\"108.162.192.0/18\",\"190.93.240.0/20\",\"188.114.96.0/20\",\"197.234.240.0/22\",\"198.41.128.0/17\",\"162.158.0.0/15\",\"104.16.0.0/13\",\"104.24.0.0/14\",\"172.64.0.0/13\",\"131.0.72.0/22\"],\"ipv6_cidrs\":[\"2400:cb00::/32\",\"2606:4700::/32\",\"2803:f800::/32\",\"2405:b500::/32\",\"2405:8100::/32\",\"2a06:98c0::/29\",\"2c0f:f248::/32\"],\"etag\":\"38f79d050aa027e3be3865e495dcc9bc\"},\"success\":true,\"errors\":[],\"messages\":[]}" +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Start β”‚ End β”‚ Duration β”‚ State β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ 10/15/2024, 1:55:31 PM β”‚ 10/15/2024, 1:55:31 PM β”‚ 0 seconds β”‚ βœ… Success β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + + Name: wait on something-1 + Type: πŸ’€ Sleeping + Start: 10/15/2024, 1:55:31 PM + End: 10/15/2024, 1:56:31 PM + Duration: 1 minute + + Name: make a call to write that could maybe, just might, fail-1 + Type: 🎯 Step + Start: 10/15/2024, 1:56:31 PM + End: 10/15/2024, 1:56:32 PM + Duration: 1 second + Success: βœ… Yes + Output: null +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Start β”‚ End β”‚ Duration β”‚ State β”‚ Error β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ 10/15/2024, 1:56:31 PM β”‚ 10/15/2024, 1:56:31 PM β”‚ 0 seconds β”‚ ❌ Error β”‚ Error: API call to $STORAGE_SYSTEM failed β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ 10/15/2024, 1:56:32 PM β”‚ 10/15/2024, 1:56:32 PM β”‚ 0 seconds β”‚ βœ… Success β”‚ β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +From the output above, we can inspect: + +* The status (success, failure, running) of each step +* Any state emitted by the step +* Any `sleep` state, including when the Workflow will wake up +* Retries associated with each step +* Errors, including exception messages + +:::note + +You do not have to wait for a Workflow instance to finish executing to inspect its current status. The `wrangler workflows instances describe` sub-command will show the status of an in-progress instance, including any persisted state, if it is sleeping, and any errors or retries. This can be especially useful when debugging a Workflow during development. + +::: + +In the previous step, we also bound a Workers script to our Workflow. You can trigger a Workflow by visiting the (deployed) Workers script in a browser or with any HTTP client. + +```sh +# This must match the URL provided in step 6 +curl -s https://workflows-starter.YOUR_WORKERS_SUBDOMAIN.workers.dev/ +``` +```sh output +{"id":"16ac31e5-db9d-48ae-a58f-95b95422d0fa","details":{"status":"queued","error":null,"output":null}} +``` + +## 7. (Optional) Clean up + +You can optionally delete the Workflow, which will prevent the creation of any (all) instances by using `wrangler`: + +```sh +npx wrangler workflows delete my-workflow +``` + +Re-deploying the Workers script containing your Workflow code will re-create the Workflow. + +--- + +## Next steps + +* Learn more about [how events are passed to a Workflow](/workflows/build/events-and-parameters/). +* Learn more about binding to and triggering Workflow instances using the [Workers API](/workflows/build/workers-api/). +* Learn more about the [Rules of Workflows](/workflows/build/rules-of-workflows/) and best practices for building applications using Workflows. + +If you have any feature requests or notice any bugs, share your feedback directly with the Cloudflare team by joining the [Cloudflare Developers community on Discord](https://discord.cloudflare.com). diff --git a/src/content/docs/workflows/get-started/index.mdx b/src/content/docs/workflows/get-started/index.mdx new file mode 100644 index 000000000000000..3d5d65fb0aba4a8 --- /dev/null +++ b/src/content/docs/workflows/get-started/index.mdx @@ -0,0 +1,13 @@ +--- +title: Get started +pcx_content_type: navigation +sidebar: + order: 1 + group: + hideIndex: true + +--- + +import { DirectoryListing } from "~/components" + + diff --git a/src/content/docs/workflows/index.mdx b/src/content/docs/workflows/index.mdx new file mode 100644 index 000000000000000..a338bcded94fc6e --- /dev/null +++ b/src/content/docs/workflows/index.mdx @@ -0,0 +1,94 @@ +--- +title: Overview +order: 0 +type: overview +pcx_content_type: overview +sidebar: + order: 1 +head: + - tag: title + content: Cloudflare Workflows + +--- + +import { CardGrid, Description, Feature, LinkTitleCard, Plan, RelatedProduct } from "~/components" + + + +Build durable multi-step applications on Cloudflare Workers with Workflows. + + + + + +Workflows is a durable execution engine built on Cloudflare Workers. Workflows allow you to build multi-step applications that can automatically retry, persist state and run for minutes, hours, days, or weeks. Workflows introduces a programming model that makes it easier to build reliable, long-running tasks, observe as they progress, and programatically trigger instances based on events across your services. + +Refer to the [get started guide](/workflows/get-started/guide/) to start building with Workflows. + +*** + +## Features + + + +Define your first Workflow, understand how to compose multi-steps, and deploy to production. + + + + + +Understand best practices when writing and building applications using Workflows. + + + + + +Learn how to trigger Workflows from your Workers applications, via the REST API, and the command-line. + + + +*** + +## Related products + + + +Build serverless applications and deploy instantly across the globe for exceptional performance, reliability, and scale. + + + + + + +Deploy dynamic front-end applications in record time. + + + + +*** + +## More resources + + + + +Learn more about how Workflows is priced. + + + +Learn more about Workflow limits, and how to work within them. + + + +Learn more about the storage and database options you can build on with Workers. + + + +Connect with the Workers community on Discord to ask questions, show what you are building, and discuss the platform with other developers. + + + +Follow @CloudflareDev on Twitter to learn about product announcements, and what is new in Cloudflare Developer Platform. + + + diff --git a/src/content/docs/workflows/observability/index.mdx b/src/content/docs/workflows/observability/index.mdx new file mode 100644 index 000000000000000..bb19599ff91a034 --- /dev/null +++ b/src/content/docs/workflows/observability/index.mdx @@ -0,0 +1,13 @@ +--- +title: Observability +pcx_content_type: navigation +sidebar: + order: 5 + group: + hideIndex: true + +--- + +import { DirectoryListing } from "~/components" + + diff --git a/src/content/docs/workflows/observability/metrics-analytics.mdx b/src/content/docs/workflows/observability/metrics-analytics.mdx new file mode 100644 index 000000000000000..e67e25e73e4495d --- /dev/null +++ b/src/content/docs/workflows/observability/metrics-analytics.mdx @@ -0,0 +1,207 @@ +--- +pcx_content_type: concept +title: Metrics and analytics +sidebar: + order: 10 + +--- + +Workflows expose metrics that allow you to inspect and measure Workflow execution, error rates, steps, and total duration across each (and all) of your Workflows. + +The metrics displayed in the [Cloudflare dashboard](https://dash.cloudflare.com/) charts are queried from Cloudflare’s [GraphQL Analytics API](/analytics/graphql-api/). You can access the metrics [programmatically](#query-via-the-graphql-api) via GraphQL or HTTP client. + +## Metrics + +Workflows currently export the below metrics within the `workflowsAdaptiveGroups` GraphQL dataset. + +| Metric | GraphQL Field Name | Description | +| ---------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| Read Queries (qps) | `readQueries` | The number of read queries issued against a database. This is the raw number of read queries, and is not used for billing. | + +Metrics can be queried (and are retained) for the past 31 days. + +### Labels and dimensions + +The `workflowsAdaptiveGroups` dataset provides the following dimensions for filtering and grouping query results: + +* `workflowName` - Workflow name - e.g. `my-workflow` +* `instanceId` - Instance ID +* `stepName` - Step name +* `eventType` - Event type (see [event types](#event-types)) +* `stepCount` - Step number within a given instance +* `date` - The date when the Workflow was triggered +* `datetimeFifteenMinutes` - The date and time truncated to fifteen minutes +* `datetimeFiveMinutes` - The date and time truncated to five minutes +* `datetimeHour` - The date and time truncated to the hour +* `datetimeMinute` - The date and time truncated to the minute + +### Event types + +The `eventType` metric allows you to filter (or groupBy) Workflows and steps based on their last observed status. + +The possible values for `eventType` are documented below: + +#### Workflows-level status labels + +* `WORKFLOW_QUEUED` - the Workflow is queued, but not currently running. This can happen when you are at the [concurrency limit](/workflows/reference/limits/) and new instances are waiting for currently running instances to complete. +* `WORKFLOW_START` - the Workflow has started and is running. +* `WORKFLOW_SUCCESS` - the Workflow finished without errors. +* `WORKFLOW_FAILURE` - the Workflow failed due to errors (exhausting retries, errors thrown, etc). +* `WORKFLOW_TERMINATED` - the Workflow was explicitly terminated. + +#### Step-level status labels + +* `STEP_START` - the step has started and is running. +* `STEP_SUCCESS` - the step finished without errors. +* `STEP_FAILURE` - the step failed due to an error. +* `SLEEP_START` - the step is sleeping. +* `SLEEP_COMPLETE` - the step last finished sleeping. +* `ATTEMPT_START` - a step is retrying. +* `ATTEMPT_SUCCESS` - the retry succeeded. +* `ATTEMPT_FAILURE` - the retry attempt failed. + +## View metrics in the dashboard + +Per-Workflow and instance analytics for Workflows are available in the Cloudflare dashboard. To view current and historical metrics for a database: + +1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com) and select your account. +2. Go to [**Workers & Pages** > **Workflows**](https://dash.cloudflare.com/?to=/:account/workers/workflows). +3. Select a Workflow to view its metrics. + +You can optionally select a time window to query. This defaults to the last 24 hours. + +## Query via the GraphQL API + +You can programmatically query analytics for your Workflows via the [GraphQL Analytics API](/analytics/graphql-api/). This API queries the same datasets as the Cloudflare dashboard, and supports GraphQL [introspection](/analytics/graphql-api/features/discovery/introspection/). + +Workflows GraphQL datasets require an `accountTag` filter with your Cloudflare account ID, and includes the `workflowsAdaptiveGroups` dataset. + +### Examples + +To query the count (number of workflow invocations) and sum of `wallTime` for a given `$workflowName` between `$datetimeStart` and `$datetimeEnd`, grouping by `date`: + +```graphql +{ + viewer { + accounts(filter: { accountTag: $accountTag }) { + wallTime: workflowsAdaptiveGroups( + limit: 10000 + filter: { + datetimeHour_geq: $datetimeStart, + datetimeHour_leq: $datetimeEnd, + workflowName: $workflowName + } + orderBy: [count_DESC] + ) { + count + sum { + wallTime + } + dimensions { + date: datetimeHour + } + } + } + } +} +``` + +Here we are doing the same for `wallTime`, `instanceRuns` and `stepCount` in the same query: + +```graphql +{ + viewer { + accounts(filter: { accountTag: $accountTag }) { + instanceRuns: workflowsAdaptiveGroups( + limit: 10000 + filter: { + datetimeHour_geq: $datetimeStart + datetimeHour_leq: $datetimeEnd + workflowName: $workflowName + eventType: "WORKFLOW_START" + } + orderBy: [count_DESC] + ) { + count + dimensions { + date: datetimeHour + } + } + stepCount: workflowsAdaptiveGroups( + limit: 10000 + filter: { + datetimeHour_geq: $datetimeStart + datetimeHour_leq: $datetimeEnd + workflowName: $workflowName + eventType: "STEP_START" + } + orderBy: [count_DESC] + ) { + count + dimensions { + date: datetimeHour + } + } + wallTime: workflowsAdaptiveGroups( + limit: 10000 + filter: { + datetimeHour_geq: $datetimeStart + datetimeHour_leq: $datetimeEnd + workflowName: $workflowName + } + orderBy: [count_DESC] + ) { + count + sum { + wallTime + } + dimensions { + date: datetimeHour + } + } + } + } +} +``` + +Here lets query `workflowsAdaptive` for raw data about `$instanceId` between `$datetimeStart` and `$datetimeEnd`: + +```graphql +{ + viewer { + accounts(filter: { accountTag: $accountTag }) { + workflowsAdaptive( + limit: 100 + filter: { + datetime_geq: $datetimeStart + datetime_leq: $datetimeEnd + instanceId: $instanceId + } + orderBy: [datetime_ASC] + ) { + datetime + eventType + workflowName + instanceId + stepName + stepCount + wallTime + } + } + } +} +``` + +#### GraphQL query variables + +Example values for the query variables: + +```json +{ + "accountTag": "fedfa729a5b0ecfd623bca1f9000f0a22", + "datetimeStart": "2024-10-20T00:00:00Z", + "datetimeEnd": "2024-10-29T00:00:00Z", + "workflowName": "shoppingCart", + "instanceId": "ecc48200-11c4-22a3-b05f-88a3c1c1db81" +} +``` diff --git a/src/content/docs/workflows/reference/changelog.mdx b/src/content/docs/workflows/reference/changelog.mdx new file mode 100644 index 000000000000000..1e63decb2fa0897 --- /dev/null +++ b/src/content/docs/workflows/reference/changelog.mdx @@ -0,0 +1,15 @@ +--- +pcx_content_type: changelog +title: Changelog +changelog_file_name: + - workflows +sidebar: + order: 99 + +--- + +import { ProductChangelog } from "~/components" + +{/* */} + + diff --git a/src/content/docs/workflows/reference/glossary.mdx b/src/content/docs/workflows/reference/glossary.mdx new file mode 100644 index 000000000000000..a00fee9cc25fbf9 --- /dev/null +++ b/src/content/docs/workflows/reference/glossary.mdx @@ -0,0 +1,13 @@ +--- +title: Glossary +pcx_content_type: glossary +sidebar: + order: 10 + +--- + +import { Glossary } from "~/components" + +Review the definitions for terms used across Cloudflare's Workflows documentation. + + diff --git a/src/content/docs/workflows/reference/index.mdx b/src/content/docs/workflows/reference/index.mdx new file mode 100644 index 000000000000000..8516bf0f828bd96 --- /dev/null +++ b/src/content/docs/workflows/reference/index.mdx @@ -0,0 +1,11 @@ +--- +pcx_content_type: navigation +title: Platform +sidebar: + order: 8 + +--- + +import { DirectoryListing } from "~/components" + + diff --git a/src/content/docs/workflows/reference/limits.mdx b/src/content/docs/workflows/reference/limits.mdx new file mode 100644 index 000000000000000..83e24c98bc12653 --- /dev/null +++ b/src/content/docs/workflows/reference/limits.mdx @@ -0,0 +1,34 @@ +--- +pcx_content_type: concept +title: Limits +sidebar: + order: 4 + +--- + +import { Render } from "~/components" + +Limits that apply to authoring, deploying, and running Workflows are detailed below. + +Many limits are inherited from those applied to Workers scripts and as documented in the [Workers limits](/workers/platform/limits/) documentation. + +| Feature | Workers Free | Workers Paid | +| ----------------------------------------- | ----------------------- | --------------------- | +| Workflow class definitions per script | 1MB max script size per [Worker size limits](/workers/platform/limits/#account-plan-limits) | 10MB max script size per [Worker size limits](/workers/platform/limits/#account-plan-limits) +| Total scripts per account | 100 | 500 (shared with [Worker script limits](/workers/platform/limits/#account-plan-limits) | +| Compute time per Workflow | 10 seconds | 30 seconds of [active CPU time](/workers/platform/limits/#cpu-time) | +| Duration (wall clock) per `step` | Unlimited | Unlimited - e.g. waiting on network I/O calls or querying a database | +| Maximum persisted state per step | 1MiB (2^20 bytes) | 1MiB (2^20 bytes) | +| Maximum state that can be persisted per Workflow instance | 100MB | 1GB | +| Maximum `step.sleep` duration | 365 days (1 year) [^1] | +| Maximum steps per Workflow | 256 [^1] | +| Maximum Workflow executions | 100,000 per day [shared with Workers daily limit](/workers/platform/limits/#worker-limits) | Unlimited | +| Concurrent Workflow instances (executions) | 25 | 100 [^1] | +| Retention limit for completed Workflow state | 3 days | 30 days [^2] | +| Maximum length of a Workflow ID | 64 bytes | + +[^1]: This limit will be reviewed and revised during the open beta for Workflows. Follow the [Workflows changelog](/workflows/reference/changelog/) for updates. + +[^2]: Workflow state and logs will be retained for 3 days on the Workers Free plan and for 7 days on the Workers Paid plan. + + diff --git a/src/content/docs/workflows/reference/pricing.mdx b/src/content/docs/workflows/reference/pricing.mdx new file mode 100644 index 000000000000000..a6150403e8f8bcc --- /dev/null +++ b/src/content/docs/workflows/reference/pricing.mdx @@ -0,0 +1,44 @@ +--- +pcx_content_type: concept +title: Pricing +sidebar: + order: 2 + +--- + +import { Render } from "~/components" + +# Pricing + +:::note + +Workflows is included in both the Free and Paid [Workers plans](/workers/platform/pricing/#workers). + +::: + +Workflows pricing is identical to [Workers Standard pricing](/workers/platform/pricing/#workers) and are billed on two dimensions: + +* **CPU time**: the total amount of compute (measured in milliseconds) consumed by a given Workflow. +* **Requests** (invocations): the number of Workflow invocations. [Subrequests](/workers/platform/limits/#subrequests) made from a Workflow do not incur additional request costs. + +A Workflow that is waiting on a response to an API call, paused as a result of calling `step.sleep`, or otherwise idle, does not incur CPU time. + +## Frequently Asked Questions + +Frequently asked questions related to Workflows pricing: + +### Are there additional costs for Workflows? + +No. Workflows are priced based on the same compute (CPU time) and requests (invocations) as Workers. + +### Are Workflows available on the [Workers Free](/workers/platform/pricing/#workers) plan? + +Yes. + +### How do Workflows show up on my bill? + +Workflows are billed as Workers, and share the same CPU time and request SKUs. + +### Are there any limits to Workflows? + +Refer to the published [limits](/workflows/reference/limits/) documentation. diff --git a/src/content/docs/workflows/reference/storage-options.mdx b/src/content/docs/workflows/reference/storage-options.mdx new file mode 100644 index 000000000000000..312ac6fdd1cbdd3 --- /dev/null +++ b/src/content/docs/workflows/reference/storage-options.mdx @@ -0,0 +1,8 @@ +--- +pcx_content_type: navigation +title: Choose a data or storage product +external_link: /workers/platform/storage-options/ +sidebar: + order: 90 + +--- diff --git a/src/content/docs/workflows/reference/wrangler-commands.mdx b/src/content/docs/workflows/reference/wrangler-commands.mdx new file mode 100644 index 000000000000000..7253dd6bea29f10 --- /dev/null +++ b/src/content/docs/workflows/reference/wrangler-commands.mdx @@ -0,0 +1,8 @@ +--- +pcx_content_type: navigation +title: Wrangler commands +external_link: /workers/wrangler/commands/#workflows +sidebar: + order: 80 + +--- diff --git a/src/content/docs/workflows/workflows-api.mdx b/src/content/docs/workflows/workflows-api.mdx new file mode 100644 index 000000000000000..5befbd93331dff1 --- /dev/null +++ b/src/content/docs/workflows/workflows-api.mdx @@ -0,0 +1,8 @@ +--- +pcx_content_type: navigation +title: Workflows REST API +external_link: /api/operations/wor-list-workflows +sidebar: + order: 10 + +--- diff --git a/src/content/glossary/workflows.yaml b/src/content/glossary/workflows.yaml new file mode 100644 index 000000000000000..ddf90ce1503dfee --- /dev/null +++ b/src/content/glossary/workflows.yaml @@ -0,0 +1,22 @@ +--- +productName: Workflows +entries: +- term: "Workflow" + general_definition: |- + The named Workflow definition, associated with a single Workers script. + +- term: "instance" + general_definition: |- + A specific instance (running, paused, errored) of a Workflow. A Workflow can have a potentially infinite number of instances. + +- term: "step" + general_definition: |- + A step is self-contained, individually retriable component of a Workflow. Steps may emit (optional) state that allows a Workflow to persist and continue from that step, even if a Workflow fails due to a network or infrastructure issue. A Workflow can have one or more steps up to the [step limit](/workflows/reference/limits/). + +- term: "Event" + general_definition: |- + The event that triggered the Workflow instance. A `WorkflowEvent` may contain optional parameters (data) that a Workflow can operate on. + +- term: "Durable Execution" + general_definition: |- + "Durable Execution" is a programming model that allows applications to execute reliably, automatically persist state, retry, and be resistant to errors caused by API, network or even machine/infrastructure failures. Cloudflare Workflows provide a way to build and deploy applications that align with this model. diff --git a/src/content/products/workflows.yaml b/src/content/products/workflows.yaml new file mode 100644 index 000000000000000..3ca59672433c4a6 --- /dev/null +++ b/src/content/products/workflows.yaml @@ -0,0 +1,33 @@ +name: Workflows + +product: + title: Workflows + url: /workflows/ + group: Developer platform + preview_tryout: true + +meta: + title: Cloudflare Workflows docs + description: Build durable, multi-step applications using the Workers platform + author: '@cloudflare' + +resources: + community: https://community.cloudflare.com/c/developers/workers/40 + dashboard_link: https://dash.cloudflare.com/?to=/:account/workers + learning_center: https://www.cloudflare.com/learning/serverless/what-is-serverless/ + discord: https://discord.com/invite/cloudflaredev + +externals: + - title: Workers home + url: https://workers.cloudflare.com + - title: Playground + url: https://workers.cloudflare.com/playground + - title: Pricing + url: https://workers.cloudflare.com/#plans + - title: Discord + url: https://discord.cloudflare.com + +algolia: + index: developers-cloudflare2 + apikey: 4edb0a6cef3338ff4bcfbc6b3d2db56b + product: workflows diff --git a/src/icons/workflows.svg b/src/icons/workflows.svg new file mode 100644 index 000000000000000..8bd10c77fe52bcf --- /dev/null +++ b/src/icons/workflows.svg @@ -0,0 +1 @@ + \ No newline at end of file