Skip to content

Commit f80723b

Browse files
mburke5678mburke5678
authored
Merge pull request #70420 from mburke5678/node-vpa-tune
OSDOCS7357 Docs for OCPSTRAT-802 Make VPA usable for large cluster
2 parents 9b7d992 + 97469cf commit f80723b

File tree

3 files changed

+176
-1
lines changed

3 files changed

+176
-1
lines changed

‎modules/nodes-pods-vertical-autoscaler-about.adoc

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,18 @@
66
[id="nodes-pods-vertical-autoscaler-about_{context}"]
77
= About the Vertical Pod Autoscaler Operator
88

9-
The Vertical Pod Autoscaler Operator (VPA) is implemented as an API resource and a custom resource (CR). The CR determines the actions the Vertical Pod Autoscaler Operator should take with the pods associated with a specific workload object, such as a daemon set, replication controller, and so forth, in a project.
9+
The Vertical Pod Autoscaler Operator (VPA) is implemented as an API resource and a custom resource (CR). The CR determines the actions that the VPA Operator should take with the pods associated with a specific workload object, such as a daemon set, replication controller, and so forth, in a project.
10+
11+
The VPA Operator consists of three components, each of which has its own pod in the VPA namespace:
12+
13+
Recommender::
14+
The VPA recommender monitors the current and past resource consumption and, based on this data, determines the optimal CPU and memory resources for the pods in the associated workload object.
15+
16+
Updater::
17+
The VPA updater checks if the pods in the associated workload object have the correct resources. If the resources are correct, the updater takes no action. If the resources are not correct, the updater kills the pod so that they can be recreated by their controllers with the updated requests.
18+
19+
Admission controller::
20+
The VPA admission controller sets the correct resource requests on each new pod in the associated workload object, whether the pod is new or was recreated by its controller due to the VPA updater actions.
1021

1122
You can use the default recommender or use your own alternative recommender to autoscale based on your own algorithms.
1223

‎modules/nodes-pods-vertical-autoscaler-tuning.adoc

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/nodes-vertical-autoscaler.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="nodes-pods-vertical-autoscaler-tuning_{context}"]
7+
= Performance tuning the VPA Operator
8+
9+
As a cluster administrator, you can tune the performance of your Vertical Pod Autoscaler Operator (VPA) to limit the rate at which the VPA makes requests of the Kubernetes API server and to specify the CPU and memory resources for the VPA recommender, updater, and admission controller component pods.
10+
11+
Additionally, you can configure the VPA Operator to monitor only those workloads that are being managed by a VPA custom resource (CR). By default, the VPA Operator monitors every workload in the cluster. This allows the VPA Operator to accrue and store 8 days of historical data for all workloads, which the Operator can use if a new VPA CR is created for a workload. However, this causes the VPA Operator to use significant CPU and memory, which could cause the Operator to fail, particularly on larger clusters. By configuring the VPA Operator to monitor only workloads with a VPA CR, you can save on CPU and memory resources. One trade-off is that if you have a workload that has been running, and you create a VPA CR to manage that workload, the VPA Operator does not have any historical data for that workload. As a result, the initial recommendations are not as useful as those after the workload had been running for some time.
12+
13+
These tunings allow you to ensure the VPA has sufficient resources to operate at peak efficiency and to prevent throttling and a possible delay in pod admissions.
14+
15+
You can perform the following tunings on the VPA components by editing the `VerticalPodAutoscalerController` custom resource (CR):
16+
17+
* To prevent throttling and pod admission delays, set the queries-per-second (QPS) and burst rates for VPA requests of the Kubernetes API server by using the `kube-api-qps` and `kube-api-burst` parameters.
18+
19+
* To ensure sufficient CPU and memory, set the CPU and memory requests for VPA component pods by using the standard `cpu` and `memory` resource requests.
20+
21+
* To configure the VPA Operator to monitor only workloads that are being managed by a VPA CR, set the `memory-saver` parameter to `true` for the recommender component.
22+
23+
The following example VPA controller CR sets the VPA API QPS and burts rates, configures the component pod resource requests, and sets `memory-saver` to `true` for the recommender:
24+
25+
.Example `VerticalPodAutoscalerController` CR
26+
[source,yaml]
27+
----
28+
apiVersion: autoscaling.openshift.io/v1
29+
kind: VerticalPodAutoscalerController
30+
metadata:
31+
name: default
32+
namespace: openshift-vertical-pod-autoscaler
33+
spec:
34+
deploymentOverrides:
35+
admission: <1>
36+
container:
37+
args: <2>
38+
- '--kube-api-qps=30.0'
39+
- '--kube-api-burst=40.0'
40+
resources:
41+
requests: <3>
42+
cpu: 40m
43+
memory: 40Mi
44+
recommender: <4>
45+
container:
46+
args:
47+
- '--kube-api-qps=20.0'
48+
- '--kube-api-burst=60.0'
49+
- '--memory-saver=true' <5>
50+
resources:
51+
requests:
52+
cpu: 60m
53+
memory: 60Mi
54+
updater: <6>
55+
container:
56+
args:
57+
- '--kube-api-qps=20.0'
58+
- '--kube-api-burst=80.0'
59+
resources:
60+
requests:
61+
cpu: 80m
62+
memory: 80Mi
63+
minReplicas: 2
64+
podMinCPUMillicores: 25
65+
podMinMemoryMb: 250
66+
recommendationOnly: false
67+
safetyMarginFraction: 0.15
68+
----
69+
<1> Specifies the tuning parameters for the VPA admission controller.
70+
<2> Specifies the API QPS and burst rates for the VPA admission controller.
71+
+
72+
--
73+
* `kube-api-qps`: Specifies the queries per second (QPS) limit when making requests to Kubernetes API server. The default is `5.0`.
74+
* `kube-api-burst`: Specifies the burst limit when making requests to Kubernetes API server. The default is `10.0`.
75+
--
76+
<3> Specifies the CPU and memory requests for the VPA admission controller pod.
77+
<4> Specifies the tuning parameters for the VPA recommender.
78+
<5> Specifies that the VPA Operator monitors only workloads with a VPA CR. The default is `false`.
79+
<6> Specifies the tuning parameters for the VPA updater.
80+
81+
You can verify that the settings were applied to each VPA component pod.
82+
83+
.Example updater pod
84+
[source,yaml]
85+
----
86+
apiVersion: v1
87+
kind: Pod
88+
metadata:
89+
name: vpa-updater-default-d65ffb9dc-hgw44
90+
namespace: openshift-vertical-pod-autoscaler
91+
# ...
92+
spec:
93+
containers:
94+
- args:
95+
- --logtostderr
96+
- --v=1
97+
- --min-replicas=2
98+
- --kube-api-qps=20.0
99+
- --kube-api-burst=80.0
100+
# ...
101+
resources:
102+
requests:
103+
cpu: 80m
104+
memory: 80Mi
105+
# ...
106+
----
107+
108+
.Example admission controller pod
109+
[source,yaml]
110+
----
111+
apiVersion: v1
112+
kind: Pod
113+
metadata:
114+
name: vpa-admission-plugin-default-756999448c-l7tsd
115+
namespace: openshift-vertical-pod-autoscaler
116+
# ...
117+
spec:
118+
containers:
119+
- args:
120+
- --logtostderr
121+
- --v=1
122+
- --tls-cert-file=/data/tls-certs/tls.crt
123+
- --tls-private-key=/data/tls-certs/tls.key
124+
- --client-ca-file=/data/tls-ca-certs/service-ca.crt
125+
- --webhook-timeout-seconds=10
126+
- --kube-api-qps=30.0
127+
- --kube-api-burst=40.0
128+
# ...
129+
resources:
130+
requests:
131+
cpu: 40m
132+
memory: 40Mi
133+
# ...
134+
----
135+
136+
.Example recommender pod
137+
[source,yaml]
138+
----
139+
apiVersion: v1
140+
kind: Pod
141+
metadata:
142+
name: vpa-recommender-default-74c979dbbc-znrd2
143+
namespace: openshift-vertical-pod-autoscaler
144+
# ...
145+
spec:
146+
containers:
147+
- args:
148+
- --logtostderr
149+
- --v=1
150+
- --recommendation-margin-fraction=0.15
151+
- --pod-recommendation-min-cpu-millicores=25
152+
- --pod-recommendation-min-memory-mb=250
153+
- --kube-api-qps=20.0
154+
- --kube-api-burst=60.0
155+
- --memory-saver=true
156+
# ...
157+
resources:
158+
requests:
159+
cpu: 60m
160+
memory: 60Mi
161+
# ...
162+
----

‎nodes/pods/nodes-pods-vertical-autoscaler.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ include::modules/nodes-pods-vertical-autoscaler-install.adoc[leveloffset=+1]
2424

2525
include::modules/nodes-pods-vertical-autoscaler-using-about.adoc[leveloffset=+1]
2626

27+
include::modules/nodes-pods-vertical-autoscaler-tuning.adoc[leveloffset=+2]
28+
2729
include::modules/nodes-pods-vertical-autoscaler-custom.adoc[leveloffset=+2]
2830

2931
include::modules/nodes-pods-vertical-autoscaler-configuring.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)