Skip to content

Commit 83fe2b7

Browse files
committed
Updates the concurrency page
1 parent 78e1be3 commit 83fe2b7

File tree

1 file changed

+33
-25
lines changed

1 file changed

+33
-25
lines changed

docs/queue-concurrency.mdx

Lines changed: 33 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,10 @@ It's important to note that only actively executing runs count towards concurren
1414
By default, all tasks have an unbounded concurrency limit, limited only by the overall concurrency limits of your environment. This means that each task could possibly "fill up" the entire
1515
concurrency limit of your environment.
1616

17+
Each individual queue has a maximum concurrency limit equal to your environment's base concurrency limit. If you don't explicitly set a queue's concurrency limit, it will default to your environment's base concurrency limit.
18+
1719
<Note>
18-
Your environment has a maximum concurrency limit which depends on your plan. If you're a paying
19-
customer you can request a higher limit by [contacting us](https://www.trigger.dev/contact).
20+
Your environment has a base concurrency limit and a burstable limit (default burst factor of 2.0x the base limit). Individual queues are limited by the base concurrency limit, not the burstable limit. For example, if your base limit is 10, your environment can burst up to 20 concurrent runs, but any single queue can have at most 10 concurrent runs. If you're a paying customer you can request higher limits by [contacting us](https://www.trigger.dev/contact).
2021
</Note>
2122

2223
## Setting task concurrency
@@ -182,12 +183,21 @@ export const subtask = task({
182183

183184
## Waits and concurrency
184185

185-
With our [task checkpoint system](/how-it-works#the-checkpoint-resume-system), a parent task can trigger and wait for a subtask to complete. The way this system interacts with the concurrency system is a little complicated but important to understand. There are two main scenarios that we handle slightly differently:
186+
With our [task checkpoint system](/how-it-works#the-checkpoint-resume-system), tasks can wait at various waitpoints (like waiting for subtasks to complete, delays, or external events). The way this system interacts with the concurrency system is important to understand.
187+
188+
Concurrency is only released when a run reaches a waitpoint and is checkpointed. When a run is checkpointed, it transitions to the `WAITING` state and releases its concurrency slot back to both the queue and the environment, allowing other runs to execute or resume.
186189

187-
- When a parent task waits for a subtask on a different queue.
188-
- When a parent task waits for a subtask on the same queue.
190+
This means that:
191+
- Only actively executing runs count towards concurrency limits
192+
- Runs in the `WAITING` state (checkpointed at waitpoints) do not consume concurrency slots
193+
- You can have more runs in the `WAITING` state than your queue's concurrency limit
194+
- When a waiting run resumes (e.g., when a subtask completes), it must re-acquire a concurrency slot
189195

190-
These scenarios are discussed in more detail below:
196+
For example, if you have a queue with a `concurrencyLimit` of 1:
197+
- You can only have exactly 1 run executing at a time
198+
- You may have multiple runs in the `WAITING` state that belong to that queue
199+
- When the executing run reaches a waitpoint and checkpoints, it releases its slot
200+
- The next queued run can then begin execution
191201

192202
<Note>
193203
We sometimes refer to the parent task as the "parent" and the subtask as the "child". Subtask and
@@ -196,7 +206,7 @@ These scenarios are discussed in more detail below:
196206

197207
### Waiting for a subtask on a different queue
198208

199-
During the time when a parent task is waiting on a subtask, the "concurrency" slot of the parent task is still considered occupied on the parent task queue, but is temporarily "released" to the environment. An example will help illustrate this:
209+
When a parent task triggers and waits for a subtask on a different queue, the parent task will checkpoint and release its concurrency slot once it reaches the wait point. This prevents environment deadlocks where all concurrency slots would be occupied by waiting tasks.
200210

201211
```ts /trigger/waiting.ts
202212
export const parentTask = task({
@@ -205,8 +215,10 @@ export const parentTask = task({
205215
concurrencyLimit: 1,
206216
},
207217
run: async (payload) => {
208-
//trigger a subtask
218+
//trigger a subtask and wait for it to complete
209219
await subtask.triggerAndWait(payload);
220+
// The parent task checkpoints here and releases its concurrency slot
221+
// allowing other tasks to execute while waiting
210222
},
211223
});
212224

@@ -218,17 +230,11 @@ export const subtask = task({
218230
});
219231
```
220232

221-
For example purposes, let's say the environment concurrency limit is 1. When the parent task is triggered, it will occupy the only slot in the environment. When the parent task triggers the subtask, the subtask will be placed in the queue for the subtask. The parent task will then wait for the subtask to complete. During this time, the parent task slot is temporarily released to the environment, allowing another task to run. Once the subtask completes, the parent task slot is reoccupied.
222-
223-
This system prevents "stuck" tasks. If the parent task were to wait on the subtask and not release the slot, the environment would be stuck with only one task running.
224-
225-
And because only the environment slot is released, the parent task queue slot is still occupied. This means that if another task is triggered on the parent task queue, it will be placed in the queue and wait for the parent task to complete, respecting the concurrency limit.
233+
When the parent task reaches the `triggerAndWait` call, it checkpoints and transitions to the `WAITING` state, releasing its concurrency slot back to both its queue and the environment. Once the subtask completes, the parent task will resume and re-acquire a concurrency slot.
226234

227235
### Waiting for a subtask on the same queue
228236

229-
Because tasks can trigger and wait recursively, or share the same queue, we've added special handling for when a parent task waits for a subtask on the same queue.
230-
231-
Recall above that when waiting for a subtask on a different queue, the parent task slot is temporarily released to the environment. When the parent task and the subtask share a queue, we also release the parent task slot to the queue. Again, an example will help illustrate this:
237+
When a parent task and subtask share the same queue, the checkpointing behavior ensures that recursive task execution can proceed without deadlocks, up to the queue's concurrency limit.
232238

233239
```ts /trigger/waiting-same-queue.ts
234240
export const myQueue = queue({
@@ -240,7 +246,7 @@ export const parentTask = task({
240246
id: "parent-task",
241247
queue: myQueue,
242248
run: async (payload) => {
243-
//trigger a subtask
249+
//trigger a subtask and wait for it to complete
244250
await subtask.triggerAndWait(payload);
245251
},
246252
});
@@ -254,9 +260,9 @@ export const subtask = task({
254260
});
255261
```
256262

257-
In this example, the parent task and the subtask share the same queue with a concurrency limit of 1. When the parent task triggers the subtask, the parent task slot is released to the queue, giving the subtask the opportunity to run. Once the subtask completes, the parent task slot is reoccupied.
263+
When the parent task checkpoints at the `triggerAndWait` call, it releases its concurrency slot back to the queue, allowing the subtask to execute. Once the subtask completes, the parent task will resume.
258264

259-
It's very important to note that we only release at-most X slots to the queue, where X is the concurrency limit of the queue. This means that you can only trigger and wait for X subtasks on the same queue. If you try to trigger and wait for more than X subtasks, you will receive a `RECURSIVE_WAIT_DEADLOCK` error. The following example will result in a deadlock:
265+
However, you can only have recursive waits up to your queue's concurrency limit. If you exceed this limit, you will receive a `RECURSIVE_WAIT_DEADLOCK` error:
260266

261267
```ts /trigger/deadlock.ts
262268
export const myQueue = queue({
@@ -268,7 +274,6 @@ export const parentTask = task({
268274
id: "parent-task",
269275
queue: myQueue,
270276
run: async (payload) => {
271-
//trigger a subtask
272277
await subtask.triggerAndWait(payload);
273278
},
274279
});
@@ -277,8 +282,7 @@ export const subtask = task({
277282
id: "subtask",
278283
queue: myQueue,
279284
run: async (payload) => {
280-
//trigger a subtask
281-
await subsubtask.triggerAndWait(payload);
285+
await subsubtask.triggerAndWait(payload); // This will cause a deadlock
282286
},
283287
});
284288

@@ -291,12 +295,16 @@ export const subsubtask = task({
291295
});
292296
```
293297

294-
Now this will result in a `RECURSIVE_WAIT_DEADLOCK` error because the parent task is waiting for the subtask, and the subtask is waiting for the subsubtask, but there is no more concurrency available in the queue. It will look a bit like this in the logs:
298+
This results in a `RECURSIVE_WAIT_DEADLOCK` error because the queue can only support one level of recursive waiting with a concurrency limit of 1:
295299

296300
![Recursive task deadlock](/images/recursive-task-deadlock-min.png)
297301

298302
### Mitigating recursive wait deadlocks
299303

300-
If you are recursively triggering and waiting for tasks on the same queue, you can mitigate the risk of a deadlock by increasing the concurrency limit of the queue. This will allow you to trigger and wait for more subtasks.
304+
To avoid recursive wait deadlocks when using shared queues:
305+
306+
1. **Increase the queue's concurrency limit** to allow more levels of recursive waiting
307+
2. **Use different queues** for parent and child tasks to eliminate the possibility of deadlock
308+
3. **Design task hierarchies** to minimize deep recursive waiting patterns
301309

302-
You can also use different queues for the parent task and the subtask. This will allow you to trigger and wait for more subtasks without the risk of a deadlock.
310+
Remember that the number of recursive waits you can have on a shared queue is limited by that queue's concurrency limit.

0 commit comments

Comments
 (0)