diff --git a/docs/migrating-from-v3.mdx b/docs/migrating-from-v3.mdx index 726b0721d5..e06cd08064 100644 --- a/docs/migrating-from-v3.mdx +++ b/docs/migrating-from-v3.mdx @@ -290,7 +290,7 @@ const batchHandle = await tasks.batchTrigger([ console.log(batchHandle.runs); ``` -In v4, you now need to use the `runs.list()` method to get the list of runs: +In v4, you now need to use the `batch.retrieve()` method to get the batch with its runs: ```ts // In v4 @@ -299,9 +299,9 @@ const batchHandle = await tasks.batchTrigger([ [myOtherTask, { baz: "qux" }], ]); -// Now you need to call runs.list() -const runs = await batchHandle.runs.list(); -console.log(runs); +// Now you need to retrieve the batch to get the runs +const batch = await batch.retrieve(batchHandle.batchId); +console.log(batch.runs); ``` ### OpenTelemetry diff --git a/docs/snippets/migrate-v4-using-ai.mdx b/docs/snippets/migrate-v4-using-ai.mdx index 75f7507edf..1681bd30fc 100644 --- a/docs/snippets/migrate-v4-using-ai.mdx +++ b/docs/snippets/migrate-v4-using-ai.mdx @@ -200,8 +200,8 @@ const batchHandle = await tasks.batchTrigger([ [myOtherTask, { baz: "qux" }], ]); -const runs = await batchHandle.runs.list(); // Use runs.list() -console.log(runs); +const batch = await batch.retrieve(batchHandle.batchId); // Use batch.retrieve() +console.log(batch.runs); Can you help me convert the following code from v3 to v4? Please include the full converted code in the answer, do not truncate it anywhere. diff --git a/docs/tags.mdx b/docs/tags.mdx index 66b66a010e..3d89bfb78c 100644 --- a/docs/tags.mdx +++ b/docs/tags.mdx @@ -75,7 +75,7 @@ export const myTask = task({ }); ``` -Reminder: you can only have up to 5 tags per run. If you call `tags.add()` and the total number of tags will be more than 5 we log an error and ignore the new tags. That includes tags from triggering and from inside the run function. +Reminder: you can only have up to 10 tags per run. If you call `tags.add()` and the total number of tags will be more than 10 we log an error and ignore the new tags. That includes tags from triggering and from inside the run function. ### Propagating tags to child runs diff --git a/docs/upgrade-to-v4.mdx b/docs/upgrade-to-v4.mdx deleted file mode 100644 index 97c1f3e4d1..0000000000 --- a/docs/upgrade-to-v4.mdx +++ /dev/null @@ -1,1214 +0,0 @@ ---- -title: "Upgrading to v4" -description: "What's new in v4, how to upgrade, and breaking changes." ---- - -import NodeVersions from "/snippets/node-versions.mdx"; - -## What's new in v4? - -[Read our blog post](https://trigger.dev/blog/v4-beta-launch) for an overview of the new features. - -### Node.js support - - - -### Wait tokens - -In addition to waiting for a specific duration, or waiting for a child task to complete, you can now create and wait for a token to be completed, giving you more flexibility and the ability to wait for arbitrary conditions. For example, you can send the token to a Slack channel, and only complete the token when the user has clicked an "Approve" button. - -To wait for a token, you need to first create one using the `wait.createToken` function: - -```ts -import { wait } from "@trigger.dev/sdk"; - -// Somewhere in your code, either your backend or inside a task -const token = await wait.createToken({ - timeout: "10m", // you can optionally specify a timeout for the token -}); - -await sendTokenToSlack(token.id); -``` - -Wait tokens are completed with a payload that you can specify when you complete the token: - -```ts -// When the user clicks the "Approve" button, you can complete the token -await wait.completeToken(tokenId, { - status: "approved", -}); -``` - -You can wait for the token using the token ID: - -```ts -type ApprovalToken = { - status: "approved" | "rejected"; -}; - -// Inside a task -const result = await wait.forToken(tokenId); - -if (result.ok) { - console.log("Token completed", result.output.status); // "approved" or "rejected" -} else { - console.log("Token timed out", result.error); -} -``` - -### Wait idempotency - -You can now pass an idempotency key to any wait function, allowing you to skip waits if the same idempotency key is used again. This can be useful if you want to skip waits when retrying a task, for example: - -```ts -// Specify the idempotency key and TTL when creating a wait token -const token = await wait.createToken({ - idempotencyKey: "my-idempotency-key", - idempotencyKeyTTL: "1h", -}); - -// Specify the idempotency key and TTL when waiting for a duration: -await wait.for({ seconds: 10 }, { idempotencyKey: "my-idempotency-key", idempotencyKeyTTL: "1h" }); - -// Specify the idempotency key and TTL when waiting for a child task: -await childTask.triggerAndWait( - { foo: "bar" }, - { - idempotencyKey: "my-idempotency-key", - idempotencyKeyTTL: "1h", - } -); -``` - -`idempotencyKeyTTL` allows you to specify how long the idempotency key should be valid for. The default is 30 days. - -### Priority - -You can now specify a priority when triggering a task. This allows you to prioritize certain tasks over others, and is useful if you want to ensure that certain tasks are executed before others. - -```ts -await task.trigger({ foo: "bar" }, { priority: 1 }); -``` - -The priority value is a time duration in seconds, which offsets the timestamp of the run in the queue. If you specify a priority of `10`, the run will win over runs with a priority of `0` that were triggered within the last 10 seconds. A more concrete example: - -```ts -// Triggered at 12:00:00, into a queue with a large number of queued runs -await task.trigger({ foo: "bar" }, { priority: 0 }); -// Triggered at 12:00:09, into the same queue -await task.trigger({ foo: "bar" }, { priority: 10 }); -``` - -In this case, the second run will be executed first, because it's priority moved it 1 second ahead of the first run. - - - We purposefully chose to use a time duration as the priority value instead of specifying priority - levels, because priority levels can cause "level starvation" where lower priority runs are never - executed because there are always higher priority runs in the queue. - - -### Global lifecycle hooks - -We've added a new way to register global lifecycle hooks that are executed for all runs, regardless of the task. Previously, this was only possible in the `trigger.config.ts` file, but now you can register them anywhere in your codebase: - -```ts -import { tasks } from "@trigger.dev/sdk"; - -tasks.onStart(({ ctx, payload, task }) => { - console.log("Run started", ctx.run); -}); - -tasks.onSuccess(({ ctx, output }) => { - console.log("Run finished", ctx.run); -}); - -tasks.onFailure(({ ctx, error }) => { - console.log("Run failed", ctx.run); -}); -``` - -### `init.ts` - -If you create a `init.ts` file at the root of your trigger directory, it will be automatically loaded when a task is executed. This is useful if you want to register global lifecycle hooks, or initialize a database connection, etc. - -```ts init.ts -import { tasks } from "@trigger.dev/sdk"; - -tasks.onStart(({ ctx, payload, task }) => { - console.log("Run started", ctx.run); -}); -``` - -### onWait and onResume - -We've added two new lifecycle hooks that allow you to run code when a run is paused or resumed because of a wait: - -```ts -export const myTask = task({ - id: "my-task", - onWait: async ({ wait }) => { - console.log("Run paused", wait); - }, - onResume: async ({ wait }) => { - console.log("Run resumed", wait); - }, - run: async (payload: any, { ctx }) => { - console.log("Run started", ctx.run); - - await wait.for({ seconds: 10 }); - - console.log("Run finished", ctx.run); - }, -}); -``` - -### onComplete - -We've added a new lifecycle hook that is executed when a run completes, regardless of whether it succeeded or failed: - -```ts -tasks.onComplete(({ ctx, result }) => { - if (result.ok) { - console.log("Run succeeded", result.data); - } else { - console.log("Run failed", result.error); - } -}); -``` - -### onCancel - -Available in v4.0.0-beta.12 and later. - -You can now define an `onCancel` hook that is called when a run is cancelled. This is useful if you want to clean up any resources that were allocated for the run. - -```ts -tasks.onCancel(({ ctx, signal }) => { - console.log("Run cancelled", signal); -}); -``` - -You can use the `onCancel` hook along with the `signal` passed into the run function to interrupt a call to an external service, for example using the [streamText](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text) function from the AI SDK: - -```ts -import { logger, tasks, schemaTask } from "@trigger.dev/sdk"; -import { streamText } from "ai"; -import { z } from "zod"; - -export const interruptibleChat = schemaTask({ - id: "interruptible-chat", - description: "Chat with the AI", - schema: z.object({ - prompt: z.string().describe("The prompt to chat with the AI"), - }), - run: async ({ prompt }, { signal }) => { - const chunks: TextStreamPart<{}>[] = []; - - // 👇 This is a global onCancel hook, but it's inside of the run function - tasks.onCancel(async () => { - // We have access to the chunks here, and can save them to the database - await saveChunksToDatabase(chunks); - }); - - try { - const result = streamText({ - model: getModel(), - prompt, - experimental_telemetry: { - isEnabled: true, - }, - tools: {}, - abortSignal: signal, // 👈 Pass the signal to the streamText function, which aborts with the run is cancelled - onChunk: ({ chunk }) => { - chunks.push(chunk); - }, - }); - - const textParts = []; - - for await (const part of result.textStream) { - textParts.push(part); - } - - return textParts.join(""); - } catch (error) { - if (error instanceof Error && error.name === "AbortError") { - // streamText will throw an AbortError if the signal is aborted, so we can handle it here - } else { - throw error; - } - } - }, -}); -``` - -The `onCancel` hook can optionally wait for the `run` function to finish, and access the output of the run: - -```ts -import { logger, task } from "@trigger.dev/sdk"; -import { setTimeout } from "node:timers/promises"; - -export const cancelExampleTask = task({ - id: "cancel-example", - // Signal will be aborted when the task is cancelled 👇 - run: async (payload: { message: string }, { signal }) => { - try { - // We pass the signal to setTimeout to abort the timeout if the task is cancelled - await setTimeout(10_000, undefined, { signal }); - } catch (error) { - // Ignore the abort error - } - - // Do some more work here - - return { - message: "Hello, world!", - }; - }, - onCancel: async ({ runPromise }) => { - // You can await the runPromise to get the output of the task - const output = await runPromise; - }, -}); -``` - - - You will have up to 30 seconds to complete the `runPromise` in the `onCancel` hook. After that - point the process will be killed. - - -### Improved middleware and locals - -Our task middleware system is now much more useful. Previously it only ran "around" the `run` function, but now we've hoisted it to the top level and it now runs before/after all the other hooks. - -We've also added a new `locals` API that allows you to share data between middleware and hooks. - -```ts db.ts -import { locals } from "@trigger.dev/sdk"; -import { logger, tasks } from "@trigger.dev/sdk"; - -// This would be type of your database client here -const DbLocal = locals.create<{ connect: () => Promise; disconnect: () => Promise }>( - "db" -); - -export function getDb() { - return locals.getOrThrow(DbLocal); -} - -export function setDb(db: { connect: () => Promise }) { - locals.set(DbLocal, db); -} - -tasks.middleware("db", async ({ ctx, payload, next, task }) => { - // This would be your database client here - const db = locals.set(DbLocal, { - connect: async () => { - logger.info("Connecting to the database"); - }, - disconnect: async () => { - logger.info("Disconnecting from the database"); - }, - }); - - await db.connect(); - - await next(); - - await db.disconnect(); -}); - -// Disconnect when the run is paused -tasks.onWait("db", async ({ ctx, payload, task }) => { - const db = getDb(); - await db.disconnect(); -}); - -// Reconnect when the run is resumed -tasks.onResume("db", async ({ ctx, payload, task }) => { - const db = getDb(); - await db.connect(); -}); -``` - -Now in your tasks `run` function and all your hooks (global or task specific) you can access the database client using `getDb()`: - -```ts -import { getDb } from "./db"; - -export const myTask = task({ - run: async (payload: any, { ctx }) => { - const db = getDb(); - await db.query("SELECT 1"); - }, -}); -``` - -### Hidden tasks - -Previously, you were required to export the task from a file in your trigger directory to be able to execute it. We've changed the way tasks are indexed and this requirement has been removed. So you can now just define a task without exporting it, and everything will still work: - -```ts trigger/my-task.ts -import { task } from "@trigger.dev/sdk"; - -const myTask = task({ - run: async (payload: any, { ctx }) => {}, -}); -``` - -You can use this to define "hidden" tasks that should only ever be triggered by other tasks in the same file: - -```ts trigger/my-task.ts -import { task } from "@trigger.dev/sdk"; - -const myTask = task({ - run: async (payload: any, { ctx }) => {}, -}); - -export const myTask2 = task({ - run: async (payload: any, { ctx }) => { - await myTask.trigger(payload); - }, -}); -``` - -Or you can create a package of reusable tasks that can be imported and used in your tasks, without having to re-export them: - -```ts trigger/my-task.ts -import { task } from "@trigger.dev/sdk"; -import { sendToSlack } from "@repo/tasks"; - -export const myTask = task({ - run: async (payload: any, { ctx }) => { - await sendToSlack.trigger(payload); - }, -}); -``` - -### useWaitToken - -We've added a new `useWaitToken` react hook that allows you to complete a wait token from a React component, using a Public Access Token. - -```ts backend.ts -import { wait } from "@trigger.dev/sdk"; - -// Somewhere in your code, you'll need to create the token and then pass the token ID and the public token to the frontend -const token = await wait.createToken({ - timeout: "10m", -}); - -return { - tokenId: token.id, - publicToken: token.publicAccessToken, // An automatically generated public access token that expires in 1 hour -}; -``` - -Now you can use the `useWaitToken` hook in your frontend code: - -```tsx frontend.tsx -import { useWaitToken } from "@trigger.dev/react-hooks"; - -export function MyComponent({ publicToken, tokenId }: { publicToken: string; tokenId: string }) { - const { complete } = useWaitToken(tokenId, { - accessToken: publicToken, - }); - - return ; -} -``` - -### `ai.tool` - -We've added a new `ai.tool` function that allows you to create an AI tool from an existing `schemaTask` to use with the Vercel [AI SDK](https://vercel.com/docs/ai-sdk): - -```ts -import { ai } from "@trigger.dev/sdk/ai"; -import { schemaTask } from "@trigger.dev/sdk"; -import { z } from "zod"; -import { generateText } from "ai"; - -const myToolTask = schemaTask({ - id: "my-tool-task", - schema: z.object({ - foo: z.string(), - }), - run: async (payload: any, { ctx }) => {}, -}); - -const myTool = ai.tool(myToolTask); - -export const myAiTask = schemaTask({ - id: "my-ai-task", - schema: z.object({ - text: z.string(), - }), - run: async (payload, { ctx }) => { - const { text } = await generateText({ - prompt: payload.text, - model: openai("gpt-4o"), - tools: { - myTool, - }, - }); - }, -}); -``` - -You can also pass the `experimental_toToolResultContent` option to the `ai.tool` function to customize the content of the tool result: - -```ts -import { openai } from "@ai-sdk/openai"; -import { Sandbox } from "@e2b/code-interpreter"; -import { ai } from "@trigger.dev/sdk/ai"; -import { schemaTask } from "@trigger.dev/sdk/v3"; -import { generateObject } from "ai"; -import { z } from "zod"; - -const chartTask = schemaTask({ - id: "chart", - description: "Generate a chart using natural language", - schema: z.object({ - input: z.string().describe("The chart to generate"), - }), - run: async ({ input }) => { - const code = await generateObject({ - model: openai("gpt-4o"), - schema: z.object({ - code: z.string().describe("The Python code to execute"), - }), - system: ` - You are a helpful assistant that can generate Python code to be executed in a sandbox, using matplotlib.pyplot. - - For example: - - import matplotlib.pyplot as plt - plt.plot([1, 2, 3, 4]) - plt.ylabel('some numbers') - plt.show() - - Make sure the code ends with plt.show() - `, - prompt: input, - }); - - const sandbox = await Sandbox.create(); - - const execution = await sandbox.runCode(code.object.code); - - const firstResult = execution.results[0]; - - if (firstResult.png) { - return { - chart: firstResult.png, - }; - } else { - throw new Error("No chart generated"); - } - }, -}); - -// This is useful if you want to return an image from the tool -export const chartTool = ai.tool(chartTask, { - experimental_toToolResultContent: (result) => { - return [ - { - type: "image", - data: result.chart, - mimeType: "image/png", - }, - ]; - }, -}); -``` - -You can also now get access to the current tool execution options inside the task run function using the `ai.currentToolOptions()` function: - -```ts -import { ai } from "@trigger.dev/sdk/ai"; -import { schemaTask } from "@trigger.dev/sdk"; -import { z } from "zod"; - -const myToolTask = schemaTask({ - id: "my-tool-task", - schema: z.object({ - foo: z.string(), - }), - run: async (payload, { ctx }) => { - const toolOptions = ai.currentToolOptions(); - console.log(toolOptions); - }, -}); - -export const myAiTask = ai.tool(myToolTask); -``` - -See the [AI SDK tool execution options docs](https://sdk.vercel.ai/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-options) for more details on the tool execution options. - - - `ai.tool` is compatible with `schemaTask`'s defined with Zod and ArkType schemas, or any schemas - that implement a `.toJsonSchema()` function. - - -## How to migrate to v4 - -First read the deprecations, breaking changes, and known issues sections below. - -We recommend the following steps to migrate to v4: - -1. Install the v4 package. -2. Run the `trigger dev` CLI command and test your tasks locally, fixing any breaking changes. -3. Deploy to the staging environment and test your tasks in staging, fixing any breaking changes. (this step is optional, but highly recommended) -4. Once you've verified that v4 is working as expected, you should deploy your application backend with the updated v4 package. -5. Once you've deployed your application backend, you should deploy your tasks to the production environment. - -Note that between steps 4 and 5, runs triggered with the v4 package will continue using v3, and only new runs triggered after step 5 is complete will use v4. - - - Once v4 is activated in your environment, there will be a period of time where old runs will - continue to execute using v3, while new runs will use v4. Because these engines use completely - different underlying queues and concurrency models, it's possible you may have up to double the - amount of concurrently executing runs. Once the runs drain from the old run engine, the - concurrency will return to normal. - - -## Installation - -To opt-in to using v4, you will need to update your dependencies to the latest version: - - - -```bash npm -npm add @trigger.dev/sdk@latest -E -``` - -```bash yarn -yarn add @trigger.dev/sdk@latest -E -``` - -```bash pnpm -pnpm add @trigger.dev/sdk@latest -E -``` - - - - You will need to do this for all your `@trigger.dev/*` packages. - -You'll also need to use the `latest` version of the `trigger.dev` CLI package: - - - -```bash npx -npx trigger.dev@latest dev -``` - -```bash yarn -yarn dlx trigger.dev@latest dev -``` - -```bash pnpm -pnpm dlx trigger.dev@latest dev -``` - - - -## Known issues - -During the beta we will be tracking issues and releasing regular fixes. - -## Deprecations - -We've deprecated the following APIs: - -### @trigger.dev/sdk/v3 - -We've deprecated the `@trigger.dev/sdk/v3` import path and moved to a new path: - -```ts -// This still works, but will be removed in a future version -import { task } from "@trigger.dev/sdk/v3"; - -// This is the new path -import { task } from "@trigger.dev/sdk"; -``` - -### `handleError` and `init` - -We've renamed the `handleError` hook to `catchError` to better reflect that it can catch and react to errors. `handleError` will be removed in a future version. - -`init` was previously used to initialize data used in the run function: - -```ts -import { task } from "@trigger.dev/sdk"; - -const myTask = task({ - init: async () => { - return { - myClient: new MyClient(), - }; - }, - run: async (payload: any, { ctx, init }) => { - const client = init.myClient; - await client.doSomething(); - }, -}); -``` - -This has now been deprecated in favor of the `locals` API and middleware. See the [Improved middleware and locals](#improved-middleware-and-locals) section for more details. - -### toolTask - -We've deprecated the `toolTask` function, which created both a Trigger.dev task and a tool compatible with the Vercel [AI SDK](https://vercel.com/docs/ai-sdk): - -```ts -import { toolTask, schemaTask } from "@trigger.dev/sdk"; -import { z } from "zod"; -import { generateText } from "ai"; - -const myToolTask = toolTask({ - id: "my-tool-task", - run: async (payload: any, { ctx }) => {}, -}); - -export const myAiTask = schemaTask({ - id: "my-ai-task", - schema: z.object({ - text: z.string(), - }), - run: async (payload, { ctx }) => { - const { text } = await generateText({ - prompt: payload.text, - model: openai("gpt-4o"), - tools: { - myToolTask, - }, - }); - }, -}); -``` - -We've replaced the `toolTask` function with the `ai.tool` function, which creates an AI tool from an existing `schemaTask`. See the [ai.tool](#ai-tool) section for more details. - -## Breaking changes - -### Queue changes - -Previously, it was possible to specify a queue name of a queue that did not exist, along with a concurrency limit. The queue would then be created "on-demand" with the specified concurrency limit. If the queue did exist, the concurrency limit of the queue would be updated to the specified value: - -```ts -await myTask.trigger({ foo: "bar" }, { queue: { name: "my-queue", concurrencyLimit: 10 } }); -``` - -This is no longer possible, and queues must now be defined ahead of time using the `queue` function: - -```ts -import { queue } from "@trigger.dev/sdk"; - -const myQueue = queue({ - name: "my-queue", - concurrencyLimit: 10, -}); -``` - -Now when you trigger a task, you can only specify the queue by name: - -```ts -await myTask.trigger({ foo: "bar" }, { queue: "my-queue" }); -``` - -Or you can set the queue on the task: - -```ts -import { queue, task } from "@trigger.dev/sdk"; - -const myQueue = queue({ - name: "my-queue", - concurrencyLimit: 10, -}); - -export const myTask = task({ - id: "my-task", - queue: myQueue, - run: async (payload: any, { ctx }) => {}, -}); - -// You can optionally specify the queue directly on the task -export const myTask2 = task({ - id: "my-task-2", - queue: { - name: "my-queue-2", - concurrencyLimit: 50, - }, - run: async (payload: any, { ctx }) => {}, -}); -``` - -Now you can trigger these tasks without having to specify the queue name in the trigger options: - -```ts -await myTask.trigger({ foo: "bar" }); // Will use the queue defined on the task -await myTask2.trigger({ foo: "bar" }); // Will use the queue defined on the task -``` - -### Concurrency changes - -We've changed a few things around how concurrency is managed at the environment and queue level: - -- Environment concurrency limits are now "burstable" above the base concurrency limit. The default burst factor is 2.0, meaning that the environment concurrency limit can be up to 2x the base concurrency limit. So if your base concurrency limit is 10, the environment concurrency limit can be up to 20. -- Each individual queue has a maximum concurrency limit of the environment base concurrency limit, NOT the burstable limit. So if your base concurrency limit is 10, the queue concurrency limit can be up to 10. This means if you don't set the queue concurrency limit, it will default to the environment base concurrency limit. -- The only time we "release" concurrency is when a run is checkpointed. This means that if you have a run that is waiting on a waitpoint, and the run is checkpointed, the concurrency will be released back into the queue, allowing other runs to execute/resume. We release the concurrency back to the queue and the environment. - -This means that if you have a queue with a `concurrencyLimit` of 1, you can only have exactly 1 run executing at a time, but you may have more than 1 run in the `WAITING` state that belongs to that queue. Runs are only transitioned to the `WAITING` state when they are waiting on a waitpoint and have been checkpointed. - -### New Run Statuses - -We've done some work cleaning up the run statuses. The new statuses are: - -- `PENDING_VERSION`: Task is waiting for a version update because it cannot execute without additional information (task, queue, etc.). -- `QUEUED`: Task is waiting to be executed by a worker. -- `DEQUEUED`: Task has been dequeued and is being sent to a worker to start executing. -- `EXECUTING`: Task is currently being executed by a worker. -- `WAITING`: Task has been paused by the system, and will be resumed by the system. -- `COMPLETED`: Task has been completed successfully. -- `CANCELED`: Task has been canceled by the user. -- `FAILED`: Task has failed to complete, due to an error in the task code. -- `CRASHED`: Task has crashed and won't be retried, most likely the worker ran out of resources, e.g. memory or storage. -- `SYSTEM_FAILURE`: Task has failed to complete, due to an error in the system -- `DELAYED`: Task has been scheduled to run at a specific time. -- `EXPIRED`: Task has expired and won't be executed, -- `TIMED_OUT`: Task has reached its maxDuration and has been stopped. - -We've removed the following statuses: - -- `WAITING_FOR_DEPLOY`: This is no longer used, and is replaced by `PENDING_VERSION` -- `FROZEN`: This is no longer used, and is replaced by `WAITING` -- `INTERRUPTED`: This is no longer used -- `REATTEMPTING`: This is no longer used, and is replaced by `EXECUTING` - -We've also added "boolean" helpers to runs returned via the API and from Realtime: - -- `isQueued`: Returns true when the status is `QUEUED`, `PENDING_VERSION`, or `DELAYED` -- `isExecuting`: Returns true when the status is `EXECUTING`, `DEQUEUED`. These count against your concurrency limits. -- `isWaiting`: Returns true when the status is `WAITING`. These do not count against your concurrency limits. -- `isCompleted`: Returns true when the status is any of the completed statuses. -- `isCanceled`: Returns true when the status is `CANCELED` -- `isFailed`: Returns true when the status is any of the failed statuses. -- `isSuccess`: Returns true when the status is `COMPLETED` - -### Lifecycle hooks - -We've changed the function signatures of the lifecycle hooks to be more consistent and easier to use, by unifying all the parameters into a single object that can be destructured. - -Previously, hooks received a payload as the first argument and then an additional object as the second argument: - -```ts -import { task } from "@trigger.dev/sdk"; - -export const myTask = task({ - id: "my-task", - onStart: (payload, { ctx }) => {}, - run: async (payload, { ctx }) => {}, -}); -``` - -Now, all the parameters are passed in a single object: - -```ts -import { task } from "@trigger.dev/sdk"; - -export const myTask = task({ - id: "my-task", - onStart: ({ payload, ctx }) => {}, - // The run function still uses separate parameters - run: async (payload, { ctx }) => {}, -}); -``` - -This is true for all the lifecycle hooks: - -```ts -import { task } from "@trigger.dev/sdk"; - -export const myTask = task({ - id: "my-task", - onStart: ({ payload, ctx, task }) => {}, - onSuccess: ({ payload, ctx, task, output }) => {}, - onFailure: ({ payload, ctx, task, error }) => {}, - onWait: ({ payload, ctx, task, wait }) => {}, - onResume: ({ payload, ctx, task, wait }) => {}, - onComplete: ({ payload, ctx, task, result }) => {}, - catchError: ({ payload, ctx, task, error, retry, retryAt, retryDelayInMs }) => {}, - run: async (payload, { ctx }) => {}, -}); -``` - -### Context changes - -We've made a few small changes to the `ctx` object: - -- `ctx.attempt.id` and `ctx.attempt.status` have been removed. `ctx.attempt.number` is still available. -- `ctx.task.exportName` has been removed (since we no longer require tasks to be exported to be triggered). - -### BatchTrigger changes - -The `batchTrigger` function no longer returns a `runs` list directly. In v3, you could access the runs directly from the batch handle: - -```ts -// In v3 -const batchHandle = await tasks.batchTrigger([ - [myTask, { foo: "bar" }], - [myOtherTask, { baz: "qux" }], -]); - -// You could access runs directly -console.log(batchHandle.runs); -``` - -In v4, you now need to use the `runs.list()` method to get the list of runs: - -```ts -// In v4 -const batchHandle = await tasks.batchTrigger([ - [myTask, { foo: "bar" }], - [myOtherTask, { baz: "qux" }], -]); - -// Now you need to call runs.list() -const runs = await batchHandle.runs.list(); -console.log(runs); -``` - -## v4 beta Changelog - - - [Trigger.dev v4 - release](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.0). - Please see our upgrade to v4 docs to view the full changelog. Run Engine 2.0 (alpha) - ([#1575](https://github.com/triggerdotdev/trigger.dev/pull/1575)), improved warm start times by - eagerly creating the child TaskRunProcess when a previous run as completed - ([#1879](https://github.com/triggerdotdev/trigger.dev/pull/1879)), and new lifecycle hooks - ([#1817](https://github.com/triggerdotdev/trigger.dev/pull/1817)). - - - - [Release - v4.0.0-beta.1](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.1). - Fixed init.ts in custom trigger dirs and init command now correctly installs v4-beta packages - ([#1914](https://github.com/triggerdotdev/trigger.dev/pull/1914)). Updated nypm package to support - test-based bun.lock files and handle flush errors gracefully in dev - ([#1914](https://github.com/triggerdotdev/trigger.dev/pull/1914)). - - - - [Release - v4.0.0-beta.2](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.2). - Managed run controller performance and reliability improvements - ([#1927](https://github.com/triggerdotdev/trigger.dev/pull/1927)). - - - - [Release - v4.0.0-beta.3](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.3). - Improved usage flushing ([#1931](https://github.com/triggerdotdev/trigger.dev/pull/1931)) and - fixed stalled run detection ([#1934](https://github.com/triggerdotdev/trigger.dev/pull/1934)). - - - - [Release - v4.0.0-beta.5](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.5). - Updated dependencies with various bug fixes and improvements. - - - - [Release - v4.0.0-beta.6](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.6). - The dev command now uses the platform-provided engine URL - ([#1949](https://github.com/triggerdotdev/trigger.dev/pull/1949)). - - - - [Release - v4.0.0-beta.7](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.7). - Fixed runLimiter check on dequeueRuns - ([#1953](https://github.com/triggerdotdev/trigger.dev/pull/1953)) and QUEUED status snapshot - handler ([#1963](https://github.com/triggerdotdev/trigger.dev/pull/1963)). - - - - [Release - v4.0.0-beta.8](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.8). - Prevented large outputs from overwriting each other - ([#1971](https://github.com/triggerdotdev/trigger.dev/pull/1971)). - - - - [Release - v4.0.0-beta.9](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.9). - Fixed default machine config indexing - ([#1979](https://github.com/triggerdotdev/trigger.dev/pull/1979)). - - - - [Release - v4.0.0-beta.10](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.10). - TriggerApiError 4xx errors no longer cause tasks to be retried - ([#1970](https://github.com/triggerdotdev/trigger.dev/pull/1970)). Fixed polling interval reset - bug that could create duplicate intervals, protected against unexpected attempt number changes, - and prevented run execution zombies after warm starts - ([#1987](https://github.com/triggerdotdev/trigger.dev/pull/1987)). - - - - [Release - v4.0.0-beta.11](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.11). - Updated dependencies with various bug fixes and improvements. - - - - [Release - v4.0.0-beta.12](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.12). - Display clickable links in Cursor terminal - ([#1998](https://github.com/triggerdotdev/trigger.dev/pull/1998)) and added AI assistance link - when you have build errors ([#1925](https://github.com/triggerdotdev/trigger.dev/pull/1925)). - Added validation when passing a directory to deploy command with helpful hints - ([#2013](https://github.com/triggerdotdev/trigger.dev/pull/2013)). - - - - [Release - v4.0.0-beta.13](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.13). - Correctly resolved waitpoints that come in early, ensured correct state before requesting - suspension, and fixed race conditions in snapshot processing - ([#2006](https://github.com/triggerdotdev/trigger.dev/pull/2006)). Always print full deploy logs - in CI ([#2006](https://github.com/triggerdotdev/trigger.dev/pull/2006)). - - - - [Release - v4.0.0-beta.14](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.14). - Updated dependencies with various bug fixes and improvements. - - - - [Release - v4.0.0-beta.15](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.15). - Added external log exporters and fixed missing external trace exporters in deployed tasks - ([#2038](https://github.com/triggerdotdev/trigger.dev/pull/2038)). Log images sizes for - self-hosted deploys ([#1764](https://github.com/triggerdotdev/trigger.dev/pull/1764)) and fixed - init.ts auto-import for deployed workers - ([#2041](https://github.com/triggerdotdev/trigger.dev/pull/2041)). - - - - [Release - v4.0.0-beta.16](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.16). - Fixed init.ts detection when using the sentry esbuild plugin - ([#2051](https://github.com/triggerdotdev/trigger.dev/pull/2051)). - - - - [Release - v4.0.0-beta.17](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.17). - Exposed esbuild `keepNames` ([#2091](https://github.com/triggerdotdev/trigger.dev/pull/2091)) and - `minify` options (experimental) ([#2091](https://github.com/triggerdotdev/trigger.dev/pull/2091)). - Added `experimental_autoDetectExternal` trigger config option - ([#2083](https://github.com/triggerdotdev/trigger.dev/pull/2083)) and output esbuild metafile, - which can be inspected after `deploy --dry run` - ([#2087](https://github.com/triggerdotdev/trigger.dev/pull/2087)). - - - - [Release - v4.0.0-beta.18](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.18). - Fixed dev runs ([#2094](https://github.com/triggerdotdev/trigger.dev/pull/2094)). - - - - [Release - v4.0.0-beta.19](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.19). - Added import timings and bundle size analysis - the dev command now warns about slow imports - ([#2114](https://github.com/triggerdotdev/trigger.dev/pull/2114)). Fixed metadata collapsing - correctness ([#2115](https://github.com/triggerdotdev/trigger.dev/pull/2115)), added support for - Preview branches in v4 projects ([#2086](https://github.com/triggerdotdev/trigger.dev/pull/2086)), - can now set project ref using the TRIGGER_PROJECT_REF env var - ([#2109](https://github.com/triggerdotdev/trigger.dev/pull/2109)), and fixed `syncEnvVars` for - non-preview deployments ([#2131](https://github.com/triggerdotdev/trigger.dev/pull/2131)). - - - - [Release - v4.0.0-beta.20](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.20). - Updated dependencies with various bug fixes and improvements. - - - - [Release - v4.0.0-beta.21](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.21). - Runtime agnostic SDK config via env vars - ([#2132](https://github.com/triggerdotdev/trigger.dev/pull/2132)) and fixed update command version - range handling ([#2153](https://github.com/triggerdotdev/trigger.dev/pull/2153)). Resolved issue - where CLI could get stuck during deploy finalization, unified local and remote build logic with - multi-platform build support, improved switch command which now accepts profile name as an - argument, registry configuration is now fully managed by the webapp, the deploy `--self-hosted` - flag is no longer required, and enhanced deployment error reporting and image digest retrieval - ([#2138](https://github.com/triggerdotdev/trigger.dev/pull/2138)). Updated profile switcher - ([#2150](https://github.com/triggerdotdev/trigger.dev/pull/2150)). - - - - [Release - v4.0.0-beta.22](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.22). - Fix update command version mismatch detection - ([#2199](https://github.com/triggerdotdev/trigger.dev/pull/2199)). Add project details to the - whoami command ([#2231](https://github.com/triggerdotdev/trigger.dev/pull/2231)). Serialize - metadata to prevent invalid metadata from breaking run completions - ([#2219](https://github.com/triggerdotdev/trigger.dev/pull/2219)) - -This release also includes a new experimental `processKeepAlive` option, which allows you to -keep the process alive after the run has completed for the next warm start, which makes warm starts even faster. - -Currently during a warm start, we still recreate the actual task run process between runs, leading to a completely fresh global environment for each run. This experimental option will keep the task run process alive between run executions, leading to even faster warm starts. This option is respected in both the `dev` CLI and in deployed tasks. - -To enable this option, add this to your `trigger.config.ts`: - -```ts -import { defineConfig } from "@trigger.dev/sdk"; - -export default defineConfig({ - project: "", - // This is false by default - experimental_processKeepAlive: true, - maxDuration: 60, -}); -``` - -You can also pass an object to `experimental_processKeepAlive` to provide more options: - -```ts -import { defineConfig } from "@trigger.dev/sdk"; - -export default defineConfig({ - project: "", - experimental_processKeepAlive: { - enabled: true, - // After 20 runs execute with a single process, we'll restart the process and start fresh - maxExecutionsPerProcess: 20, - // In dev, you can combine this option with setting a max pool size, giving you the ability to limit the number of processes created on your local dev machine. Has no effect on deployed tasks - devMaxPoolSize: 10, - }, - maxDuration: 60, -}); -``` - -## Gotchas - -- Be careful with memory usage and memory leaks, as this will cause memory to persist across run executions. -- It's possible different tasks get executed in the same persisted process. -- If you configure any 3rd party SDKs globally using env vars for API keys, those SDKs will not change between runs. So if you change an env var between runs, the SDK will be "stale" and continue using the old env var value. Instead, you should initialize SDKs using env vars at runtime (in any of the lifecycle hooks or inside the `run` function of a task. -- Cancelling a task will cause the task run process to be restarted. Exiting the process is the only reliable way to actually stop a running function from stopping. -- This DOES NOT effect cold starts, warm starts only will be improved. - -We recommend enabling this option and testing in a staging or preview environment before trying it out in prod, as there could be unknown issues depending on what you are doing in your tasks. [#2183](https://github.com/triggerdotdev/trigger.dev/pull/2183) - - - - - [Release - v4.0.0-beta.23](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.23). - -- fix: Logging large objects is now much more performant and uses less memory ([#2263](https://github.com/triggerdotdev/trigger.dev/pull/2263)) -- New internal idempotency implementation for trigger and batch trigger to prevent request retries from duplicating work ([#2256](https://github.com/triggerdotdev/trigger.dev/pull/2256)) - -- Enhance deploy command output to better distinguish between local and remote builds ([#2254](https://github.com/triggerdotdev/trigger.dev/pull/2254)) -- Fixes a bug that would allow processes that had OOM errors to be incorrectly reused when experimental_processKeepAlive was enabled ([#2261](https://github.com/triggerdotdev/trigger.dev/pull/2261)) -- Add runtime version detection for display in the dashboard ([#2254](https://github.com/triggerdotdev/trigger.dev/pull/2254)) -- Update base images to latest compatible versions. The `node-22` runtime now uses v22.16.0 and `bun` uses the latest v1.2.18 release. The default `node` runtime is unchanged and points at v21.7.3. ([#2254](https://github.com/triggerdotdev/trigger.dev/pull/2254)) -- Fail fast in CI when running deploy with missing `TRIGGER_ACCESS_TOKEN` and add useful error message with link to docs ([#2258](https://github.com/triggerdotdev/trigger.dev/pull/2258)) - - - - - [Release - v4.0.0-beta.24](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.24). - -- Removes the `releaseConcurrencyOnWaitpoint` option on queues and the `releaseConcurrency` option on various wait functions. Replaced with the following default behavior: ([#2284](https://github.com/triggerdotdev/trigger.dev/pull/2284)) - - - Concurrency is never released when a run is first blocked via a waitpoint, at either the env or queue level. - - Concurrency is always released when a run is checkpointed and shutdown, at both the env and queue level. - - Additionally, environment concurrency limits now have a new "Burst Factor", defaulting to 2.0x. The "Burst Factor" allows the environment-wide concurrency limit to be higher than any individual queue's concurrency limit. For example, if you have an environment concurrency limit of 100, and a Burst Factor of 2.0x, then you can execute up to 200 runs concurrently, but any one task/queue can still only execute 100 runs concurrently. - - We've done some work cleaning up the run statuses. The new statuses are: - - - `PENDING_VERSION`: Task is waiting for a version update because it cannot execute without additional information (task, queue, etc.) - - `QUEUED`: Task is waiting to be executed by a worker - - `DEQUEUED`: Task has been dequeued and is being sent to a worker to start executing. - - `EXECUTING`: Task is currently being executed by a worker - - `WAITING`: Task has been paused by the system, and will be resumed by the system - - `COMPLETED`: Task has been completed successfully - - `CANCELED`: Task has been canceled by the user - - `FAILED`: Task has failed to complete, due to an error in the system - - `CRASHED`: Task has crashed and won't be retried, most likely the worker ran out of resources, e.g. memory or storage - - `SYSTEM_FAILURE`: Task has failed to complete, due to an error in the system - - `DELAYED`: Task has been scheduled to run at a specific time - - `EXPIRED`: Task has expired and won't be executed - - `TIMED_OUT`: Task has reached it's maxDuration and has been stopped - - We've removed the following statuses: - - - `WAITING_FOR_DEPLOY`: This is no longer used, and is replaced by `PENDING_VERSION` - - `FROZEN`: This is no longer used, and is replaced by `WAITING` - - `INTERRUPTED`: This is no longer used - - `REATTEMPTING`: This is no longer used, and is replaced by `EXECUTING` - - We've also added "boolean" helpers to runs returned via the API and from Realtime: - - - `isQueued`: Returns true when the status is `QUEUED`, `PENDING_VERSION`, or `DELAYED` - - `isExecuting`: Returns true when the status is `EXECUTING`, `DEQUEUED`. These count against your concurrency limits. - - `isWaiting`: Returns true when the status is `WAITING`. These do not count against your concurrency limits. - - `isCompleted`: Returns true when the status is any of the completed statuses. - - `isCanceled`: Returns true when the status is `CANCELED` - - `isFailed`: Returns true when the status is any of the failed statuses. - - `isSuccess`: Returns true when the status is `COMPLETED` - - This change adds the ability to easily detect which runs are being counted against your concurrency limit by filtering for both `EXECUTING` or `DEQUEUED`. - -- Added runs.list filtering for queue and machine ([#2277](https://github.com/triggerdotdev/trigger.dev/pull/2277)) - - - - - [Release - v4.0.0-beta.25](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.25). - -- Gracefully shutdown task run processes using SIGTERM followed by SIGKILL after a 1s timeout. This also prevents cancelled or completed runs from leaving orphaned task run processes behind ([#2299](https://github.com/triggerdotdev/trigger.dev/pull/2299)) - - - - - [Release - v4.0.0-beta.26](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.26). - -- Allow any runs to finish after SIGTERM but disable warm starts ([#2316](https://github.com/triggerdotdev/trigger.dev/pull/2316)) -- Switch to profile after successful login ([#2192](https://github.com/triggerdotdev/trigger.dev/pull/2192)) -- Fixed importing from runEngine/index.js breaking non-node runtimes ([#2328](https://github.com/triggerdotdev/trigger.dev/pull/2328)) -- Added and cleaned up the run ctx param ([#2322](https://github.com/triggerdotdev/trigger.dev/pull/2322)): - - New optional properties `ctx.run.parentTaskRunId` and `ctx.run.rootTaskRunId` reference the current run's root/parent ID - - Removed deprecated properties from `ctx` - - Added a new `ctx.deployment` object that contains information about the deployment associated with the run - - Updated `metadata.root` and `metadata.parent` to work even when the run is a "root" run (meaning it doesn't have a parent or a root associated run) - - - - - [Release - v4.0.0-beta.27](https://github.com/triggerdotdev/trigger.dev/releases/tag/trigger.dev%404.0.0-v4-beta.27). - -- All experimental flags have been promoted to non-experimental, but the experimental ones still work (for now). `keepNames` and `autoDetectExternal` now default to true. ([#2371](https://github.com/triggerdotdev/trigger.dev/pull/2371)) -- Improve playwright non-headless chrome installation ([#2347](https://github.com/triggerdotdev/trigger.dev/pull/2347)) -- Upgrade to zod 3.25.76 ([#2352](https://github.com/triggerdotdev/trigger.dev/pull/2352)) -- External Trace Correlation & OpenTelemetry Package Updates. ([#2334](https://github.com/triggerdotdev/trigger.dev/pull/2334)) -- Add jsonSchema support when indexing tasks ([#2353](https://github.com/triggerdotdev/trigger.dev/pull/2353)) -- Removed triggerAndPoll. It was never recommended so it's been removed. ([#2379](https://github.com/triggerdotdev/trigger.dev/pull/2379)) -- Specify a region override when triggering a run ([#2366](https://github.com/triggerdotdev/trigger.dev/pull/2366)) - -