From 84adcadb9e114b4104c9dc27c0fd519f68d84a73 Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Tue, 18 Feb 2025 11:45:09 +0100 Subject: [PATCH 1/5] docs(srv): add content reference on autoscaling MTA-5516 --- .../functions-autoscaling.mdx | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 pages/serverless-functions/reference-content/functions-autoscaling.mdx diff --git a/pages/serverless-functions/reference-content/functions-autoscaling.mdx b/pages/serverless-functions/reference-content/functions-autoscaling.mdx new file mode 100644 index 0000000000..e1e1f6b205 --- /dev/null +++ b/pages/serverless-functions/reference-content/functions-autoscaling.mdx @@ -0,0 +1,50 @@ +--- +meta: + title: Functions autoscaling reference + description: Understand how autoscaling works for Serverless Functions in Scaleway. +content: + h1: Functions autoscaling reference + paragraph: Understand how autoscaling works for Serverless Functions in Scaleway. +tags: serverless functions autoscaling scale up down min max +dates: + validation: 2025-02-18 + posted: 2025-02-18 +categories: + - serverless + - functions +--- + +## Benefits of autoscaling + +## Minimum and maximum scales + +### Minimum scale (min-scale) + +The this parameter sets the lowest value your resource is allowed to scale down to: + +- If you set a value of `0`, all instances of your resource will be terminated after 15 minutes of inactivity. + +- If you set a value of `1` or more, the corresponding number of instances of your resource will remain available at all time. + +Customizing the minimum scale for Serverless can help ensure that an instance remains pre-allocated and ready to handle requests, reducing delays associated with [cold starts](/serverless-functions/concepts/#cold-start). However, this setting also impacts the costs of your Serverless Function. + +### Maximum scale (max-scale) + +This parameter sets the maximum number of instances of your resource. You should adjust it based on your function’s traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively. + +### Autoscaler behavior + +The autoscaler decides to start new instances when: + + - the existing instances are no longer able to handle the load because they are busy responding to other ongoing requests. By default, this happens if an instance is already processing 80 requests (max_concurrency = 80). + + - our system detects an unusual number of requests. In this case, some instances may be started in anticipation to avoid a potential cold start. + +The same autoscaler decides to remove instances when: + + - no more requests are being processed. If even a single request is being processed (or detected as being processed), then the autoscaler will not be able to remove this instance. The system also prioritizes instances with the fewest ongoing requests, or if very few requests are being sent, it tries to select a particular instance to shut down the others, and therefore scale down. + - an instance has not responded to a request for more than 15 minutes of inactivity. The instance is only shut down after this interval, once again to absorb any potential new peaks and thus avoid the cold start. These 15 minutes of inactivity are not configurable. + + +Redeploying your resource results in the termination of existing instances and a return to the min scale, which you observe when redeploying. + \ No newline at end of file From 0f47574b0d7d674c4fcaca6a2a3741d4caf4bb04 Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Tue, 18 Feb 2025 15:04:15 +0100 Subject: [PATCH 2/5] docs(srv): update --- .../containers-autoscaling.mdx | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 pages/serverless-containers/reference-content/containers-autoscaling.mdx diff --git a/pages/serverless-containers/reference-content/containers-autoscaling.mdx b/pages/serverless-containers/reference-content/containers-autoscaling.mdx new file mode 100644 index 0000000000..9b34796da1 --- /dev/null +++ b/pages/serverless-containers/reference-content/containers-autoscaling.mdx @@ -0,0 +1,52 @@ +--- +meta: + title: Containers autoscaling reference + description: Understand how autoscaling works for Serverless Containers in Scaleway. +content: + h1: Containers autoscaling reference + paragraph: Understand how autoscaling works for Serverless Containers in Scaleway. +tags: serverless containers autoscaling scale up down min max +dates: + validation: 2025-02-18 + posted: 2025-02-18 +categories: + - serverless + - containers +--- + +## Benefits of autoscaling + +Autoscaling offers several benefits, including improved responsiveness and cost efficiency. By automatically adjusting the number of instances of your resource based on current demand, you ensure that your applications can handle varying loads without manual intervention. This not only enhances user experience by maintaining performance levels but also helps in reducing costs by only using resources when necessary. Additionally, autoscaling helps in optimal resource utilization, minimizing the risk of performance degradation during peak times. + +## Minimum and maximum scales + +### Minimum scale (min-scale) + +The this parameter sets the lowest value your resource is allowed to scale down to: + +- If you set a value of `0`, all instances of your resource will be terminated after 15 minutes of inactivity. + +- If you set a value of `1` or more, the corresponding number of instances of your resource will remain available at all time. + +Customizing the minimum scale for Serverless can help ensure that an instance remains pre-allocated and ready to handle requests, reducing delays associated with [cold starts](/serverless-containers/concepts/#cold-start). However, this setting also impacts the costs of your Serverless Container. + +### Maximum scale (max-scale) + +This parameter sets the maximum number of instances of your resource. You should adjust it based on your resource's traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively. + +### Autoscaler behavior + +The autoscaler decides to start new instances when: + + - the existing instances are no longer able to handle the load because they are busy responding to other ongoing requests. By default, this happens if an instance is already processing 80 requests (max_concurrency = 80). + + - our system detects an unusual number of requests. In this case, some instances may be started in anticipation to avoid a potential cold start. + +The same autoscaler decides to remove instances when: + + - no more requests are being processed. If even a single request is being processed (or detected as being processed), then the autoscaler will not be able to remove this instance. The system also prioritizes instances with the fewest ongoing requests, or if very few requests are being sent, it tries to select a particular instance to shut down the others, and therefore scale down. + - an instance has not responded to a request for more than 15 minutes of inactivity. The instance is only shut down after this interval, once again to absorb any potential new peaks and thus avoid the cold start. These 15 minutes of inactivity are not configurable. + + +Redeploying your resource results in the termination of existing instances and a return to the min scale, which you observe when redeploying. + \ No newline at end of file From 48db8528f503855de9664e4796b9221a5024fabd Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Tue, 18 Feb 2025 15:00:36 +0100 Subject: [PATCH 3/5] docs(srv): update --- menu/navigation.json | 8 ++++++++ .../reference-content/functions-autoscaling.mdx | 5 +++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/menu/navigation.json b/menu/navigation.json index 42823a2293..8371bb0967 100644 --- a/menu/navigation.json +++ b/menu/navigation.json @@ -3972,6 +3972,10 @@ "label": "Functions handlers", "slug": "functions-handlers" }, + { + "label": "Functions autoscaling", + "slug": "functions-autoscaling" + }, { "label": "Code examples", "slug": "code-examples" @@ -4158,6 +4162,10 @@ "label": "Containers concurrency", "slug": "containers-concurrency" }, + { + "label": "Containers autoscaling", + "slug": "containers-autoscaling" + }, { "label": "Containers billing", "slug": "containers-billing" diff --git a/pages/serverless-functions/reference-content/functions-autoscaling.mdx b/pages/serverless-functions/reference-content/functions-autoscaling.mdx index e1e1f6b205..05409f3967 100644 --- a/pages/serverless-functions/reference-content/functions-autoscaling.mdx +++ b/pages/serverless-functions/reference-content/functions-autoscaling.mdx @@ -16,6 +16,8 @@ categories: ## Benefits of autoscaling +Autoscaling offers several benefits, including improved responsiveness and cost efficiency. By automatically adjusting the number of instances of your resource based on current demand, you ensure that your applications can handle varying loads without manual intervention. This not only enhances user experience by maintaining performance levels but also helps in reducing costs by only using resources when necessary. Additionally, autoscaling helps in optimal resource utilization, minimizing the risk of performance degradation during peak times. + ## Minimum and maximum scales ### Minimum scale (min-scale) @@ -30,14 +32,13 @@ Customizing the minimum scale for Serverless can help ensure that an instance re ### Maximum scale (max-scale) -This parameter sets the maximum number of instances of your resource. You should adjust it based on your function’s traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively. +This parameter sets the maximum number of instances of your resource. You should adjust it based on your resource's traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively. ### Autoscaler behavior The autoscaler decides to start new instances when: - the existing instances are no longer able to handle the load because they are busy responding to other ongoing requests. By default, this happens if an instance is already processing 80 requests (max_concurrency = 80). - - our system detects an unusual number of requests. In this case, some instances may be started in anticipation to avoid a potential cold start. The same autoscaler decides to remove instances when: From 5f64f9f26467bebc51b5d3da3afd47b361553d1b Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Tue, 18 Feb 2025 15:25:43 +0100 Subject: [PATCH 4/5] docs(srv): update --- .../reference-content/containers-autoscaling.mdx | 4 +++- .../reference-content/functions-autoscaling.mdx | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/pages/serverless-containers/reference-content/containers-autoscaling.mdx b/pages/serverless-containers/reference-content/containers-autoscaling.mdx index 9b34796da1..b74ae0dde4 100644 --- a/pages/serverless-containers/reference-content/containers-autoscaling.mdx +++ b/pages/serverless-containers/reference-content/containers-autoscaling.mdx @@ -32,7 +32,9 @@ Customizing the minimum scale for Serverless can help ensure that an instance re ### Maximum scale (max-scale) -This parameter sets the maximum number of instances of your resource. You should adjust it based on your resource's traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively. +This parameter sets the maximum number of instances of your resource. You should adjust it based on your resource's traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively.`` + +When the maximum scale is reached, new requests are queued for processing. When the queue is full, the resource returns `503` errors for requests received beyond this point. ### Autoscaler behavior diff --git a/pages/serverless-functions/reference-content/functions-autoscaling.mdx b/pages/serverless-functions/reference-content/functions-autoscaling.mdx index 05409f3967..eca3214e22 100644 --- a/pages/serverless-functions/reference-content/functions-autoscaling.mdx +++ b/pages/serverless-functions/reference-content/functions-autoscaling.mdx @@ -34,6 +34,8 @@ Customizing the minimum scale for Serverless can help ensure that an instance re This parameter sets the maximum number of instances of your resource. You should adjust it based on your resource's traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively. +When the maximum scale is reached, new requests are queued for processing. When the queue is full, the resource returns `503` errors for requests received beyond this point. + ### Autoscaler behavior The autoscaler decides to start new instances when: From 7af524c01a91d9691d2c70356a45c0abb6ce64db Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Fri, 21 Feb 2025 10:43:16 +0100 Subject: [PATCH 5/5] docs(srv): update --- .../reference-content/containers-autoscaling.mdx | 6 +++--- .../reference-content/functions-autoscaling.mdx | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/pages/serverless-containers/reference-content/containers-autoscaling.mdx b/pages/serverless-containers/reference-content/containers-autoscaling.mdx index b74ae0dde4..e7bed35ece 100644 --- a/pages/serverless-containers/reference-content/containers-autoscaling.mdx +++ b/pages/serverless-containers/reference-content/containers-autoscaling.mdx @@ -16,13 +16,13 @@ categories: ## Benefits of autoscaling -Autoscaling offers several benefits, including improved responsiveness and cost efficiency. By automatically adjusting the number of instances of your resource based on current demand, you ensure that your applications can handle varying loads without manual intervention. This not only enhances user experience by maintaining performance levels but also helps in reducing costs by only using resources when necessary. Additionally, autoscaling helps in optimal resource utilization, minimizing the risk of performance degradation during peak times. +[Autoscaling](/serverless-containers/concepts/#autoscaling) offers several benefits, including improved responsiveness and cost efficiency. By automatically adjusting the number of instances of your resource based on current demand, you ensure that your applications can handle varying loads without manual intervention. This not only enhances user experience by maintaining performance levels but also helps in reducing costs by only using resources when necessary. Additionally, autoscaling helps in optimal resource utilization, minimizing the risk of performance degradation during peak times. ## Minimum and maximum scales ### Minimum scale (min-scale) -The this parameter sets the lowest value your resource is allowed to scale down to: +This parameter sets the lowest value your resource is allowed to scale down to: - If you set a value of `0`, all instances of your resource will be terminated after 15 minutes of inactivity. @@ -32,7 +32,7 @@ Customizing the minimum scale for Serverless can help ensure that an instance re ### Maximum scale (max-scale) -This parameter sets the maximum number of instances of your resource. You should adjust it based on your resource's traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively.`` +This parameter sets the maximum number of instances of your resource. You should adjust it based on your resource's traffic spikes, keeping in mind that you may wish to limit the max scale to manage costs effectively. When the maximum scale is reached, new requests are queued for processing. When the queue is full, the resource returns `503` errors for requests received beyond this point. diff --git a/pages/serverless-functions/reference-content/functions-autoscaling.mdx b/pages/serverless-functions/reference-content/functions-autoscaling.mdx index eca3214e22..3db3e13935 100644 --- a/pages/serverless-functions/reference-content/functions-autoscaling.mdx +++ b/pages/serverless-functions/reference-content/functions-autoscaling.mdx @@ -16,13 +16,13 @@ categories: ## Benefits of autoscaling -Autoscaling offers several benefits, including improved responsiveness and cost efficiency. By automatically adjusting the number of instances of your resource based on current demand, you ensure that your applications can handle varying loads without manual intervention. This not only enhances user experience by maintaining performance levels but also helps in reducing costs by only using resources when necessary. Additionally, autoscaling helps in optimal resource utilization, minimizing the risk of performance degradation during peak times. +[Autoscaling](/serverless-functions/concepts/#autoscaling) offers several benefits, including improved responsiveness and cost efficiency. By automatically adjusting the number of instances of your resource based on current demand, you ensure that your applications can handle varying loads without manual intervention. This not only enhances user experience by maintaining performance levels but also helps in reducing costs by only using resources when necessary. Additionally, autoscaling helps in optimal resource utilization, minimizing the risk of performance degradation during peak times. ## Minimum and maximum scales ### Minimum scale (min-scale) -The this parameter sets the lowest value your resource is allowed to scale down to: +This parameter sets the lowest value your resource is allowed to scale down to: - If you set a value of `0`, all instances of your resource will be terminated after 15 minutes of inactivity.