From da857a03d379c1f0c3acaa1ef1eb26d435f8a0de Mon Sep 17 00:00:00 2001 From: fpagny Date: Fri, 21 Mar 2025 16:26:51 +0100 Subject: [PATCH 1/2] feat(genapi): update rate limits --- .../reference-content/rate-limits.mdx | 24 ++++++++++++------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/pages/generative-apis/reference-content/rate-limits.mdx b/pages/generative-apis/reference-content/rate-limits.mdx index 809babe1da..f880605679 100644 --- a/pages/generative-apis/reference-content/rate-limits.mdx +++ b/pages/generative-apis/reference-content/rate-limits.mdx @@ -17,25 +17,31 @@ Any model served through Scaleway Generative APIs gets limited by: - Tokens per minute - Queries per minute +Base limits apply if you registered a valid payment method. This limits are increased automatically if you also validate your identity. + -These limits only apply if you created a Scaleway Account and registered a valid payment method. Otherwise, stricter limits apply to ensure usage stays within Free Tier only. +If you created a Scaleway Account but did not register a valid payment method, stricter limits apply to ensure usage stays within Free Tier only. ## How can I increase the rate limits? We actively monitor usage and will improve rates based on feedback. -If you need to increase your rate limits, [contact our support team](https://console.scaleway.com/support/create), providing details on the model used and specific use case. +If you need to increase your rate limits: +- [Verify your identity](https://www.scaleway.com/en/docs/account/how-to/verify-identity/) to automatically increase your rate limit as described below +- [Contact our support team](https://console.scaleway.com/support/create), providing details on the model used and specific use case, for additional increase. Note that for increases of up to x5 or x10 volumes, we highly recommend using dedicated deployments with [Managed Inference](https://console.scaleway.com/inference/deployments), which provides exactly the same features and API compatibility. ### Chat models -| Model string | Requests per minute | Total tokens per minute | -|-----------------|-----------------|-----------------| -| `llama-3.1-8b-instruct` | 300 | 200K | -| `llama-3.1-70b-instruct` | 300 | 200K | -| `mistral-nemo-instruct-2407`| 300 | 200K | -| `pixtral-12b-2409`| 300 | 200K | -| `qwen2.5-32b-instruct`| 300 | 200K | +| Model string | Additional steps required | Requests per minute | Total tokens per minute | +|-----------------|-----------------|-----------------|-----------------| +| `llama-3.1-8b-instruct` | None | 300 | 200K | +| `llama-3.1-70b-instruct` | None | 300 | 200K | +| `llama-3.3-70b-instruct` | None | 300 | 200K | +| `llama-3.3-70b-instruct` | Identity verified | 600 | 400K | +| `mistral-nemo-instruct-2407`| None | 300 | 200K | +| `pixtral-12b-2409`| None | 300 | 200K | +| `qwen2.5-32b-instruct`| None | 300 | 200K | ### Embedding models From fd73d4ea0d1da99f5d10b967c81e10e3f4769d37 Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Mon, 24 Mar 2025 10:47:10 +0100 Subject: [PATCH 2/2] Update pages/generative-apis/reference-content/rate-limits.mdx Co-authored-by: Rowena Jones <36301604+RoRoJ@users.noreply.github.com> --- pages/generative-apis/reference-content/rate-limits.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/generative-apis/reference-content/rate-limits.mdx b/pages/generative-apis/reference-content/rate-limits.mdx index f880605679..4600b4bc97 100644 --- a/pages/generative-apis/reference-content/rate-limits.mdx +++ b/pages/generative-apis/reference-content/rate-limits.mdx @@ -27,7 +27,7 @@ If you created a Scaleway Account but did not register a valid payment method, s We actively monitor usage and will improve rates based on feedback. If you need to increase your rate limits: -- [Verify your identity](https://www.scaleway.com/en/docs/account/how-to/verify-identity/) to automatically increase your rate limit as described below +- [Verify your identity](/account/how-to/verify-identity/) to automatically increase your rate limit as described below - [Contact our support team](https://console.scaleway.com/support/create), providing details on the model used and specific use case, for additional increase. Note that for increases of up to x5 or x10 volumes, we highly recommend using dedicated deployments with [Managed Inference](https://console.scaleway.com/inference/deployments), which provides exactly the same features and API compatibility.