Skip to content

Commit 3426891

Browse files
authored
feat: Improve docs around rate limiting, mention higher rate limit for key-value store record endpoints (#927)
The documentation around rate limits was a bit confusing, so I expanded it a bit, to make the distinction between global and per-resource rate limits clearer. I've also added a mention of the increased rate limits on key-value store records.
1 parent 084818b commit 3426891

File tree

1 file changed

+24
-6
lines changed

1 file changed

+24
-6
lines changed

sources/platform/api_v2/api_v2_reference.apib

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -324,9 +324,27 @@ Here is the table of the most common errors that can occur for many API endpoint
324324
## Rate limiting
325325

326326
All API endpoints limit the rate of requests in order to prevent overloading of Apify servers by misbehaving clients.
327-
The default rate limit is 30 requests per second and is applied to every API endpoint except the following list
328-
of endpoints that are rate limited to 200 requests per second:
329327

328+
There are two kinds of rate limits - a global rate limit and a per-resource rate limit.
329+
330+
### Global rate limit
331+
332+
The global rate limit is set to _250 000 requests per minute_.
333+
For [authenticated]((#authentication) requests, it is counted per user,
334+
and for unauthenticated requests, it is counted per IP address.
335+
336+
### Per-resource rate limit
337+
The default per-resource rate limit is _30 requests per second per resource_, which in this context means a single Actor, a single Actor run, a single dataset, single key-value store etc.
338+
The default rate limit is applied to every API endpoint except a few select ones, which have higher rate limits.
339+
Each API endpoint returns its rate limit in `X-RateLimit-Limit` header.
340+
341+
These endpoints have a rate limit of _100 requests per second per resource_:
342+
* CRUD ([get](#reference/key-value-stores/record/get-record),
343+
[put](#reference/key-value-stores/record/put-record),
344+
[delete](#reference/key-value-stores/record/delete-record))
345+
operations on key-value store records
346+
347+
These endpoints have a rate limit of _200 requests per second per resource_:
330348
* [Run actor](#reference/actors/run-collection/run-actor)
331349
* [Run actor task asynchronously](#reference/actor-tasks/runs-collection/run-task-asynchronously)
332350
* [Run actor task synchronously](#reference/actor-tasks/runs-collection/run-task-synchronously)
@@ -337,11 +355,9 @@ of endpoints that are rate limited to 200 requests per second:
337355
[get](#reference/request-queues/request-collection/get-request),
338356
[update](#reference/request-queues/request-collection/update-request),
339357
[delete](#reference/request-queues/request-collection/delete-request))
340-
operations of request queue requests
358+
operations on requests in request queues
341359

342-
> For authenticated API endpoints (if you use an [authentication token](#authentication)), the limit is counted per user.
343-
Storage API endpoints (when a token isn't used) count the limit per store ID.
344-
Anonymous API endpoints count the limit per client IP adress. Each API endpoint returns its rate limit in `X-RateLimit-Limit` header.
360+
### Rate limit exceeded errors
345361

346362
If the client is sending too many requests, the API endpoints respond with the HTTP status code `429 Too Many Requests`
347363
and the following body:
@@ -355,6 +371,8 @@ and the following body:
355371
}
356372
```
357373

374+
### Retrying rate-limited requests with exponential backoff
375+
358376
If the client receives the rate limit error, it should wait a certain period of time and then retry the request.
359377
If the error happens again, the client should double the wait period and retry the request,
360378
and so on. This algorithm is known as _exponential backoff_

0 commit comments

Comments
 (0)