From 6a5ca27a6f85b64a27d919c0ce3146d07adeba58 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Fri, 4 Apr 2025 14:33:58 -0500 Subject: [PATCH 1/6] Note in ECK autoscaling docs how cpu/ram is scaled. Signed-off-by: Michael Montgomery --- deploy-manage/autoscaling/autoscaling-in-eck.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/deploy-manage/autoscaling/autoscaling-in-eck.md b/deploy-manage/autoscaling/autoscaling-in-eck.md index cd27a422a1..8885508055 100644 --- a/deploy-manage/autoscaling/autoscaling-in-eck.md +++ b/deploy-manage/autoscaling/autoscaling-in-eck.md @@ -18,7 +18,7 @@ Elasticsearch autoscaling requires a valid Enterprise license or Enterprise tria :::: -ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. +ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. ECK scales Elasticsearch data and machine learning tiers exclusively by scaling storage. CPU and Memory are scaled *relative* to the storage resource min/max settings, and not independently. ### Enable autoscaling [k8s-enable] @@ -100,7 +100,7 @@ In the case of storage the following restrictions apply: #### Scale Up and Scale Out [k8s-autoscaling-algorithm] -In order to adapt the resources to the workload, the operator first attempts to scale up the resources (cpu, memory, and storage) allocated to each node in the NodeSets. The operator always ensures that the requested resources are within the limits specified in the autoscaling policy. If each individual node has reached the limits specified in the autoscaling policy, but more resources are required to handle the load, then the operator adds some nodes to the NodeSets. Nodes are added up to the `max` value specified in the `nodeCount` of the policy. +In order to adapt the resources to the workload, the operator first attempts to scale up the resources (storage, with cpu and ram relative to storage) allocated to each node in the NodeSets. The operator always ensures that the requested resources are within the limits specified in the autoscaling policy. If each individual node has reached the limits specified in the autoscaling policy, but more resources are required to handle the load, then the operator adds some nodes to the NodeSets. Nodes are added up to the `max` value specified in the `nodeCount` of the policy. ::::{warning} Scaling up (vertically) is only supported if the actual storage capacity of the persistent volumes matches the capacity claimed. If the physical capacity of a PersistentVolume may be greater than the capacity claimed in the PersistentVolumeClaim, it is advised to set the same value for the `min` and the `max` setting of each resource. It is however still possible to let the operator scale out the NodeSets automatically, as in the following example: @@ -345,4 +345,4 @@ spec: target: type: Utilization averageUtilization: 50 -``` \ No newline at end of file +``` From 7b6bb38f45a393e496e9954953e4896b5b86b56a Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Mon, 7 Apr 2025 13:40:03 -0500 Subject: [PATCH 2/6] Further clarify the autoscaling settings. Signed-off-by: Michael Montgomery --- deploy-manage/autoscaling/autoscaling-in-eck.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/deploy-manage/autoscaling/autoscaling-in-eck.md b/deploy-manage/autoscaling/autoscaling-in-eck.md index 8885508055..595f68a182 100644 --- a/deploy-manage/autoscaling/autoscaling-in-eck.md +++ b/deploy-manage/autoscaling/autoscaling-in-eck.md @@ -18,7 +18,7 @@ Elasticsearch autoscaling requires a valid Enterprise license or Enterprise tria :::: -ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. ECK scales Elasticsearch data and machine learning tiers exclusively by scaling storage. CPU and Memory are scaled *relative* to the storage resource min/max settings, and not independently. +ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. ECK scales Elasticsearch data tiers exclusively by scaling storage. CPU and Memory are scaled *relative* to the storage resource min/max settings, and not independently in data tiers. ECK can scale memory and CPU on ML tiers if specified in the `ElasticsearchAutoscaler.spec`. On Frozen tiers ECK can scale memory if specified in the `ElasticsearchAutoscaler.cpu`, but will scale CPU in relation to the storage. ### Enable autoscaling [k8s-enable] @@ -100,7 +100,7 @@ In the case of storage the following restrictions apply: #### Scale Up and Scale Out [k8s-autoscaling-algorithm] -In order to adapt the resources to the workload, the operator first attempts to scale up the resources (storage, with cpu and ram relative to storage) allocated to each node in the NodeSets. The operator always ensures that the requested resources are within the limits specified in the autoscaling policy. If each individual node has reached the limits specified in the autoscaling policy, but more resources are required to handle the load, then the operator adds some nodes to the NodeSets. Nodes are added up to the `max` value specified in the `nodeCount` of the policy. +In order to adapt the resources to the workload, the operator first attempts to scale up the resources (cpu, memory, and storage) allocated to each node in the NodeSets. The operator always ensures that the requested resources are within the limits specified in the autoscaling policy. If each individual node has reached the limits specified in the autoscaling policy, but more resources are required to handle the load, then the operator adds some nodes to the NodeSets. Nodes are added up to the `max` value specified in the `nodeCount` of the policy. ::::{warning} Scaling up (vertically) is only supported if the actual storage capacity of the persistent volumes matches the capacity claimed. If the physical capacity of a PersistentVolume may be greater than the capacity claimed in the PersistentVolumeClaim, it is advised to set the same value for the `min` and the `max` setting of each resource. It is however still possible to let the operator scale out the NodeSets automatically, as in the following example: From d06a3cc697a1a06c882b3644ae24ff3a9136f3db Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Thu, 10 Apr 2025 09:06:05 -0500 Subject: [PATCH 3/6] Note exclusion of frozen tiers when discussing data tiers. Signed-off-by: Michael Montgomery --- deploy-manage/autoscaling/autoscaling-in-eck.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/autoscaling/autoscaling-in-eck.md b/deploy-manage/autoscaling/autoscaling-in-eck.md index 595f68a182..0519cc24d2 100644 --- a/deploy-manage/autoscaling/autoscaling-in-eck.md +++ b/deploy-manage/autoscaling/autoscaling-in-eck.md @@ -18,7 +18,7 @@ Elasticsearch autoscaling requires a valid Enterprise license or Enterprise tria :::: -ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. ECK scales Elasticsearch data tiers exclusively by scaling storage. CPU and Memory are scaled *relative* to the storage resource min/max settings, and not independently in data tiers. ECK can scale memory and CPU on ML tiers if specified in the `ElasticsearchAutoscaler.spec`. On Frozen tiers ECK can scale memory if specified in the `ElasticsearchAutoscaler.cpu`, but will scale CPU in relation to the storage. +ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in Elasticsearch 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for Elasticsearch [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. ECK scales Elasticsearch data tiers (excluding frozen tiers) exclusively by scaling storage. CPU and Memory are scaled *relative* to the storage resource min/max settings, and not independently in data tiers (again excluding frozen tiers). ECK can scale memory and CPU on ML tiers if specified in the `ElasticsearchAutoscaler.spec`. On Frozen tiers ECK can scale memory if specified in the `ElasticsearchAutoscaler.cpu`, but will scale CPU in relation to the storage. ### Enable autoscaling [k8s-enable] From 3a2be276ae8058ff69764011e780d34c51f87655 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Wed, 16 Apr 2025 14:14:03 -0500 Subject: [PATCH 4/6] Move to a table to display data more clearly. Signed-off-by: Michael Montgomery --- deploy-manage/autoscaling/autoscaling-in-eck.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/deploy-manage/autoscaling/autoscaling-in-eck.md b/deploy-manage/autoscaling/autoscaling-in-eck.md index bcfc883dae..66af8831dd 100644 --- a/deploy-manage/autoscaling/autoscaling-in-eck.md +++ b/deploy-manage/autoscaling/autoscaling-in-eck.md @@ -18,7 +18,15 @@ Configure autoscaling for {{es}} deployments in {{eck}}. Learn how to enable aut :::: -ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in {{es}} 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for {{es}} [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. ECK scales {{es}} data tiers (excluding frozen tiers) exclusively by scaling storage. CPU and Memory are scaled *relative* to the storage resource min/max settings, and not independently in data tiers (again excluding frozen tiers). ECK can scale memory and CPU on ML tiers if specified in the `ElasticsearchAutoscaler.spec`. On Frozen tiers ECK can scale memory if specified in the `ElasticsearchAutoscaler.cpu`, but will scale CPU in relation to the storage. +ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in {{es}} 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. + +### Supported Resources for Autoscaling per Elasticsearch Tier + +| Tiers | Storage | Memory | Cpu | +| --- | ---| --- | --- | +| Data Nodes (except Frozen) | Yes | Calculated proportionally to the required amount of storage | Calculated proportionally to the required amount of memory +| Frozen Nodes | Yes | Yes | Calculated proportionally to the required amount of memory +| Machine Learning | No | Yes | Calculated proportionally to the required amount of memory ### Enable autoscaling [k8s-enable] From 305914e092ae85d151353e4a0a19a7d4907acacc Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Wed, 16 Apr 2025 14:15:18 -0500 Subject: [PATCH 5/6] Add back sentence. Signed-off-by: Michael Montgomery --- deploy-manage/autoscaling/autoscaling-in-eck.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/autoscaling/autoscaling-in-eck.md b/deploy-manage/autoscaling/autoscaling-in-eck.md index 66af8831dd..e35636bd1c 100644 --- a/deploy-manage/autoscaling/autoscaling-in-eck.md +++ b/deploy-manage/autoscaling/autoscaling-in-eck.md @@ -18,7 +18,7 @@ Configure autoscaling for {{es}} deployments in {{eck}}. Learn how to enable aut :::: -ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in {{es}} 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. +ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-autoscaling) introduced in {{es}} 7.11 to adjust automatically the number of Pods and the allocated resources in a tier. Currently, autoscaling is supported for {{es}} [data tiers](/manage-data/lifecycle/data-tiers.md) and machine learning nodes. ### Supported Resources for Autoscaling per Elasticsearch Tier From f8a82dc939373beade131eb07b6f7ba06c0da7d3 Mon Sep 17 00:00:00 2001 From: Michael Montgomery Date: Thu, 17 Apr 2025 08:39:30 -0500 Subject: [PATCH 6/6] capitalize CPU in table Co-authored-by: Peter Brachwitz --- deploy-manage/autoscaling/autoscaling-in-eck.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deploy-manage/autoscaling/autoscaling-in-eck.md b/deploy-manage/autoscaling/autoscaling-in-eck.md index e35636bd1c..e9bdefce9e 100644 --- a/deploy-manage/autoscaling/autoscaling-in-eck.md +++ b/deploy-manage/autoscaling/autoscaling-in-eck.md @@ -22,7 +22,7 @@ ECK can leverage the [autoscaling API](https://www.elastic.co/docs/api/doc/elast ### Supported Resources for Autoscaling per Elasticsearch Tier -| Tiers | Storage | Memory | Cpu | +| Tiers | Storage | Memory | CPU | | --- | ---| --- | --- | | Data Nodes (except Frozen) | Yes | Calculated proportionally to the required amount of storage | Calculated proportionally to the required amount of memory | Frozen Nodes | Yes | Yes | Calculated proportionally to the required amount of memory