You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Updates 'Trained model autoscaling' page for Serverless adaptive resources behavior (elastic#2224)
On Serverless, adaptive resources are now always enabled.
This PR updates the Trained model autoscaling page to reflect this
behavior.
Related issue: elastic/developer-docs-team#309
This is a follow-up PR for
elastic#2184.
---------
Co-authored-by: Vlada Chirmicci <[email protected]>
Copy file name to clipboardExpand all lines: deploy-manage/autoscaling/trained-model-autoscaling.md
+6-12Lines changed: 6 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,11 +22,13 @@ There are two ways to enable autoscaling:
22
22
* through APIs by enabling adaptive allocations
23
23
* in {{kib}} by enabling adaptive resources
24
24
25
+
For {{serverless-short}} projects, trained model autoscaling is automatically enabled and cannot be disabled.
26
+
25
27
::::{important}
26
28
To fully leverage model autoscaling in {{ech}}, {{ece}}, and {{eck}}, it is highly recommended to enable [{{es}} deployment autoscaling](../../deploy-manage/autoscaling.md).
27
29
::::
28
30
29
-
Trained model autoscaling is available for {{serverless-short}}, {{ech}}, {{ece}}, and {{eck}} deployments. In serverless deployments, processing power is managed differently across Search, Observability, and Security projects, which impacts their costs and resource limits.
31
+
Trained model autoscaling is available for {{serverless-short}}, {{ech}}, {{ece}}, and {{eck}} deployments. In {{serverless-short}} projects, processing power is managed differently across Search, Observability, and Security projects, which impacts their costs and resource limits.
30
32
31
33
:::{admonition} Trained model auto-scaling for self-managed deployments
32
34
The available resources of self-managed deployments are static, so trained model autoscaling is not applicable. However, available resources are still segmented based on the settings described in this section.
@@ -54,10 +56,6 @@ You can enable adaptive allocations by using:
54
56
55
57
If the new allocations fit on the current {{ml}} nodes, they are immediately started. If more resource capacity is needed for creating new model allocations, then your {{ml}} node will be scaled up if {{ml}} autoscaling is enabled to provide enough resources for the new allocation. The number of model allocations can be scaled down to 0. They cannot be scaled up to more than 32 allocations, unless you explicitly set the maximum number of allocations to more. Adaptive allocations must be set up independently for each deployment and [{{infer}} endpoint](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-inference).
56
58
57
-
:::{note}
58
-
When you create inference endpoints on {{serverless-short}} using {{kib}}, adaptive allocations are automatically turned on, and there is no option to disable them.
59
-
:::
60
-
61
59
### Optimizing for typical use cases [optimizing-for-typical-use-cases]
62
60
63
61
You can optimize your model deployment for typical use cases, such as search and ingest. When you optimize for ingest, the throughput will be higher, which increases the number of {{infer}} requests that can be performed in parallel. When you optimize for search, the latency will be lower during search processes.
@@ -73,16 +71,16 @@ You can choose from three levels of resource usage for your trained model deploy
73
71
74
72
Refer to the tables in the [Model deployment resource matrix](#model-deployment-resource-matrix) section to find out the settings for the level you selected.
The image below shows the process of starting a trained model on an {{ech}} deployment. In {{serverless-short}} projects, the **Adaptive resources** toggle is not available when starting trained model deployments, as adaptive allocations are always enabled and cannot be disabled.
:alt: ELSER deployment with adaptive resources enabled.
78
78
:screenshot:
79
79
:width: 500px
80
80
:::
81
81
82
82
In {{serverless-full}}, Search projects are given access to more processing resources, while Security and Observability projects have lower limits. This difference is reflected in the UI configuration: Search projects have higher resource limits compared to Security and Observability projects to accommodate their more complex operations.
83
83
84
-
On {{serverless-short}}, adaptive allocations are automatically enabled for all project types.
85
-
86
84
## Model deployment resource matrix [model-deployment-resource-matrix]
87
85
88
86
The used resources for trained model deployments depend on three factors:
@@ -100,10 +98,6 @@ If you use a self-managed cluster or ECK, vCPUs level ranges are derived from th
100
98
101
99
The following tables show you the number of allocations, threads, and vCPUs available in ECE and ECH when adaptive resources are enabled or disabled.
102
100
103
-
::::{note}
104
-
On {{serverless-short}}, adaptive allocations are automatically enabled for all project types. However, the "Adaptive resources" control is not displayed in {{kib}} for Observability and Security projects.
105
-
::::
106
-
107
101
### Ingest optimized
108
102
109
103
In case of ingest-optimized deployments, we maximize the number of model allocations.
0 commit comments