Skip to content

feat(genapi): add maximum concurrent requests #4789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 4, 2025
Merged

feat(genapi): add maximum concurrent requests #4789

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8c0f5a0
feat(genapi): add maximum concurrent requests
fpagny Apr 4, 2025
1e23582
Update pages/organizations-and-projects/additional-content/organizati…
nerda-codes Apr 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions pages/organizations-and-projects/additional-content/organization-quotas.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,7 @@ Managed Inference Deployments are limited to a maximum number of nodes, dependin
Generative APIs are rate limited based on:
- Tokens per minute (total input and output tokens)
- Requests per minute
- Concurrent requests (total active HTTP session at the same time)

<Message type="important">
[Contact our support team](https://console.scaleway.com/support/create) if you want to increase your quotas above these limits.
Expand All @@ -194,6 +195,9 @@ Generative APIs are rate limited based on:
| qwen2.5-32b-instruct | 300 | 300 |
| bge-multilingual-gemma2 | 300 | 300 |

| Concurrent requests | [Payment method validated](/billing/how-to/add-payment-method/#how-to-add-a-credit-card) | Payment method and [identity validated](/account/how-to/verify-identity/) |
|-------------|:----------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------:|
| All models | 25 | 25 |

## Apple silicon

Expand Down