You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I made some changes in docs regarding the API changes.
* stress out that locking works on the client level and the same as on
the run level.
* update code example by separating code into two actors using code tabs
---------
Co-authored-by: Michał Olender <[email protected]>
Copy file name to clipboardExpand all lines: sources/platform/storage/request_queue.md
+79-28Lines changed: 79 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -407,7 +407,20 @@ You can lock a request so that no other clients receive it when they fetch the q
407
407
This feature is seamlessly integrated into Crawlee, requiring minimal extra setup. By default, requests are locked for the same duration as the timeout for processing requests in the crawler ([`requestHandlerTimeoutSecs`](https://crawlee.dev/api/next/basic-crawler/interface/BasicCrawlerOptions#requestHandlerTimeoutSecs)).
408
408
If the Actor processing the request fails, the lock expires, and the request is processed again eventually. For more details, refer to the [Crawlee documentation](https://crawlee.dev/docs/next/experiments/experiments-request-locking).
409
409
410
-
In the following example, we demonstrate how we can use locking mechanisms to avoid concurrent processing of the same request.
410
+
In the following example, we demonstrate how you can use locking mechanisms to avoid concurrent processing of the same request across multiple Actor runs.
411
+
412
+
:::info
413
+
The lock mechanism works on the client level, as well as the run level, when running the Actor on the Apify platform.
414
+
415
+
This means you can unlock or prolong the lock the locked request only if:
416
+
417
+
- You are using the same client key, or
418
+
- The operation is being called from the same Actor run.
419
+
420
+
:::
421
+
422
+
<TabsgroupId="main">
423
+
<TabItemvalue="Actor 1"label="Actor 1">
411
424
412
425
```js
413
426
import { Actor, ApifyClient } from'apify';
@@ -422,15 +435,12 @@ const client = new ApifyClient({
console.log(`Request locked until ${requestLockedByClientOne?.lockExpiresAt}`);
532
+
533
+
console.log(`Was the request locked by the first run locked by the second run? ${wasBothRunsLockedSameRequest}`);
534
+
console.log(`Request locked until ${requestLockedByAnotherRunDetail?.lockExpiresAt}`);
478
535
479
536
// Other clients cannot modify the lock; attempting to do so will throw an error.
480
537
try {
481
-
await requestQueueClientTwo.prolongRequestLock(
482
-
theFirstRequestLockedByClientOne.id,
538
+
await requestQueueClient.prolongRequestLock(
539
+
requestLockedByAnotherRunDetail.id,
483
540
{ lockSecs: 60 },
484
541
);
485
542
} catch (err) {
486
543
// This will throw an error.
487
544
}
488
545
489
-
// Prolongs the lock of the first request or unlocks it.
490
-
await requestQueueClientOne.prolongRequestLock(
491
-
theFirstRequestLockedByClientOne.id,
492
-
{ lockSecs: 60 },
493
-
);
494
-
await requestQueueClientOne.deleteRequestLock(
495
-
theFirstRequestLockedByClientOne.id,
496
-
);
497
-
498
546
// Cleans up the queue.
499
-
await requestQueueClientOne.delete();
547
+
await requestQueueClient.delete();
500
548
501
549
await Actor.exit();
502
550
```
503
551
552
+
</TabItem>
553
+
</Tabs>
554
+
504
555
A detailed tutorial on how to process one request queue with multiple Actor runs can be found in [Academy tutorials](https://docs.apify.com/academy/node-js/multiple-runs-scrape).
0 commit comments