Skip to content

Commit 7a7ae43

Browse files
committed
Clarify behaviour of scheduled jobs with concurrency controls
Closes #597
1 parent 5e0586f commit 7a7ae43

File tree

1 file changed

+26
-1
lines changed

1 file changed

+26
-1
lines changed

README.md

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -433,7 +433,9 @@ In the case of recurring tasks, if such error is raised when enqueuing the job c
433433

434434
## Concurrency controls
435435

436-
Solid Queue extends Active Job with concurrency controls, that allows you to limit how many jobs of a certain type or with certain arguments can run at the same time. When limited in this way, by default, jobs will be blocked from running, and they'll stay blocked until another job finishes and unblocks them, or after the set expiry time (concurrency limit's _duration_) elapses. Alternatively, jobs can be configured to be discarded instead of blocked. This means that if a job with certain arguments has already been enqueued, other jobs with the same characteristics (in the same concurrency _class_) won't be enqueued.
436+
Solid Queue extends Active Job with concurrency controls, that allows you to limit how many jobs of a certain type or with certain arguments can run at the same time. When limited in this way, **by default, jobs will be blocked from running**, and they'll stay blocked until another job finishes and unblocks them, or after the set expiry time (concurrency limit's _duration_) elapses.
437+
438+
**Alternatively, jobs can be configured to be discarded instead of blocked**. This means that if a job with certain arguments has already been enqueued, other jobs with the same characteristics (in the same concurrency _class_) won't be enqueued.
437439

438440
```ruby
439441
class MyJob < ApplicationJob
@@ -496,6 +498,29 @@ Jobs are unblocked in order of priority but **queue order is not taken into acco
496498

497499
Finally, failed jobs that are automatically or manually retried work in the same way as new jobs that get enqueued: they get in the queue for getting an open semaphore, and whenever they get it, they'll be run. It doesn't matter if they had already gotten an open semaphore in the past.
498500

501+
### Scheduled jobs
502+
503+
Jobs set to run in the future (via Active Job's `wait` or `wait_until` options) have concurrency limits enforced when they're due, not when they're scheduled. For example, consider this job:
504+
```ruby
505+
class DeliverAnnouncementToContactJob < ApplicationJob
506+
limits_concurrency to: 1, key: ->(contact) { contact.account }, duration: 5.minutes
507+
508+
def perform(contact)
509+
# ...
510+
```
511+
512+
If several jobs are enqueued like this:
513+
514+
```ruby
515+
DeliverAnnouncementToContactJob.set(wait: 10.minutes).perform_later(contact)
516+
DeliverAnnouncementToContactJob.set(wait: 10.minutes).perform_later(contact)
517+
DeliverAnnouncementToContactJob.set(wait: 30.minutes).perform_later(contact)
518+
```
519+
520+
The 3 jobs will go into the scheduled queue and will wait there until they're due. Then, 10 minutes after, the first two jobs will be enqueued and the second one most likely will be blocked because the first one will be running first. Then, assuming the jobs are fast and finish in a few seconds, when the third job is due, it'll be enqueued normally.
521+
522+
Normally scheduled jobs are enqueued in batches, but with concurrency controls, jobs need to be enqueued one by one. This has an impact on performance, similarly to the impact of concurrency controls in bulk enqueuing. Read below for more details. I'd generally advise against mixing concurrency controls with waiting/scheduling in the future.
523+
499524
### Performance considerations
500525

501526
Concurrency controls introduce significant overhead (blocked executions need to be created and promoted to ready, semaphores need to be created and updated) so you should consider carefully whether you need them. For throttling purposes, where you plan to have `limit` significantly larger than 1, I'd encourage relying on a limited number of workers per queue instead. For example:

0 commit comments

Comments
 (0)