Skip to content

Commit ce3ed04

Browse files
committed
addressing PR comments
1 parent adc263a commit ce3ed04

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

modules/configuring-metric-based-autoscaling.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@
44
= Configuring metrics-based autoscaling
55

66
[role="_abstract"]
7-
While Knative-based autoscaling features are not available in standard deployment modes, you can enable metrics-based autoscaling for an inference service in these deployments. This capability helps you efficiently manage accelerator resources, lower operational costs, and ensure that your inference services meet performance requirements.
7+
Knative-based autoscaling is not available in standard deployment mode. However, you can enable metrics-based autoscaling for an inference service in standard deployment mode. Metrics-based autoscaling helps you efficiently manage accelerator resources, lower operational costs, and ensure that your inference services meet performance requirements.
88

9-
To set up autoscaling for your inference service in standard deployments, you must install and configure the OpenShift Custom Metrics Autoscaler (CMA), which is based on Kubernetes Event-driven Autoscaling (KEDA). You can then utilize various model runtime metrics available in OpenShift Monitoring, such as KVCache utilization, Time to First Token (TTFT), and Concurrency, to trigger autoscaling of your inference service.
9+
To set up autoscaling for your inference service in standard deployments, install and configure the OpenShift Custom Metrics Autoscaler (CMA), which is based on Kubernetes Event-driven Autoscaling (KEDA). You can then use various model runtime metrics available in OpenShift Monitoring to trigger autoscaling of your inference service, such as KVCache utilization, Time to First Token (TTFT), and Concurrency.
1010

1111
.Prerequisites
1212
* You have cluster administrator privileges for your {openshift-platform} cluster.

0 commit comments

Comments
 (0)