Skip to content

Commit 676c865

Browse files
RaviHariRaviHari
authored
docs: vpa for rollouts (argoproj#1909)
Signed-off-by: Ravi Hari <[email protected]>
1 parent acf8054 commit 676c865

File tree

2 files changed

+357
-0
lines changed

2 files changed

+357
-0
lines changed

‎docs/features/vpa-support.md

Lines changed: 356 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,356 @@
1+
2+
# Vertical Pod Autoscaling
3+
4+
Vertical Pod Autoscaling (VPA) reduces the maintenance cost and improve utilization of cluster resources by automating configuration of resource requirements.
5+
6+
## VPA modes
7+
8+
There are four modes in which VPAs operate
9+
10+
1. "Auto": VPA assigns resource requests on pod creation as well as updates them on existing pods using the preferred update mechanism. Currently this is equivalent to "Recreate" (see below). Once restart free ("in-place") update of pod requests is available, it may be used as the preferred update mechanism by the "Auto" mode.
11+
NOTE: This feature of VPA is experimental and may cause downtime for your applications.
12+
13+
1. "Recreate": VPA assigns resource requests on pod creation as well as updates them on existing pods by evicting them when the requested resources differ significantly from the new recommendation (respecting the Pod Disruption Budget, if defined). This mode should be used rarely, only if you need to ensure that the pods are restarted whenever the resource request changes. Otherwise prefer the "Auto" mode which may take advantage of restart free updates once they are available.
14+
NOTE: This feature of VPA is experimental and may cause downtime for your applications.
15+
16+
1. "Initial": VPA only assigns resource requests on pod creation and never changes them later.
17+
18+
1. "Off": VPA does not automatically change resource requirements of the pods. The recommendations are calculated and can be inspected in the VPA object.
19+
20+
21+
## Example
22+
23+
Below is an example of a Vertical Pod Autoscaler with Argo-Rollouts.
24+
25+
Rollout sample app:
26+
27+
```yaml
28+
apiVersion: argoproj.io/v1alpha1
29+
kind: Rollout
30+
metadata:
31+
name: vpa-demo-rollout
32+
namespace: test-vpa
33+
spec:
34+
replicas: 5
35+
strategy:
36+
canary:
37+
steps:
38+
- setWeight: 20
39+
- pause: {duration: 10}
40+
- setWeight: 40
41+
- pause: {duration: 10}
42+
- setWeight: 60
43+
- pause: {duration: 10}
44+
- setWeight: 80
45+
- pause: {duration: 10}
46+
revisionHistoryLimit: 10
47+
selector:
48+
matchLabels:
49+
app: vpa-demo-rollout
50+
template:
51+
metadata:
52+
labels:
53+
app: vpa-demo-rollout
54+
spec:
55+
containers:
56+
- name: vpa-demo-rollout
57+
image: ravihari/nginx:v1
58+
ports:
59+
- containerPort: 80
60+
resources:
61+
requests:
62+
cpu: "5m"
63+
memory: "5Mi"
64+
```
65+
66+
VPA configuration for Rollout sample app:
67+
68+
```yaml
69+
apiVersion: "autoscaling.k8s.io/v1beta2"
70+
kind: VerticalPodAutoscaler
71+
metadata:
72+
name: vpa-rollout-example
73+
namespace: test-vpa
74+
spec:
75+
targetRef:
76+
apiVersion: "argoproj.io/v1alpha1"
77+
kind: Rollout
78+
name: vpa-demo-rollout
79+
updatePolicy:
80+
updateMode: "Auto"
81+
resourcePolicy:
82+
containerPolicies:
83+
- containerName: '*'
84+
minAllowed:
85+
cpu: 5m
86+
memory: 5Mi
87+
maxAllowed:
88+
cpu: 1
89+
memory: 500Mi
90+
controlledResources: ["cpu", "memory"]
91+
```
92+
93+
Describe VPA when initially deployed we donot see recommendations as it will take few mins.
94+
95+
```yaml
96+
Name: kubengix-vpa
97+
Namespace: test-vpa
98+
Labels: <none>
99+
Annotations: <none>
100+
API Version: autoscaling.k8s.io/v1
101+
Kind: VerticalPodAutoscaler
102+
Metadata:
103+
Creation Timestamp: 2022-03-14T12:54:06Z
104+
Generation: 1
105+
Managed Fields:
106+
API Version: autoscaling.k8s.io/v1beta2
107+
Fields Type: FieldsV1
108+
fieldsV1:
109+
f:metadata:
110+
f:annotations:
111+
.:
112+
f:kubectl.kubernetes.io/last-applied-configuration:
113+
f:spec:
114+
.:
115+
f:resourcePolicy:
116+
.:
117+
f:containerPolicies:
118+
f:targetRef:
119+
.:
120+
f:apiVersion:
121+
f:kind:
122+
f:name:
123+
f:updatePolicy:
124+
.:
125+
f:updateMode:
126+
Manager: kubectl-client-side-apply
127+
Operation: Update
128+
Time: 2022-03-14T12:54:06Z
129+
Resource Version: 3886
130+
UID: 4ac64e4c-c84b-478e-92e4-5f072f985971
131+
Spec:
132+
Resource Policy:
133+
Container Policies:
134+
Container Name: *
135+
Controlled Resources:
136+
cpu
137+
memory
138+
Max Allowed:
139+
Cpu: 1
140+
Memory: 500Mi
141+
Min Allowed:
142+
Cpu: 5m
143+
Memory: 5Mi
144+
Target Ref:
145+
API Version: argoproj.io/v1alpha1
146+
Kind: Rollout
147+
Name: vpa-demo-rollout
148+
Update Policy:
149+
Update Mode: Auto
150+
Events: <none>
151+
```
152+
153+
After few minutes when VPA starts to process and provide recommendation:
154+
155+
```yaml
156+
Name: kubengix-vpa
157+
Namespace: test-vpa
158+
Labels: <none>
159+
Annotations: <none>
160+
API Version: autoscaling.k8s.io/v1
161+
Kind: VerticalPodAutoscaler
162+
Metadata:
163+
Creation Timestamp: 2022-03-14T12:54:06Z
164+
Generation: 2
165+
Managed Fields:
166+
API Version: autoscaling.k8s.io/v1beta2
167+
Fields Type: FieldsV1
168+
fieldsV1:
169+
f:metadata:
170+
f:annotations:
171+
.:
172+
f:kubectl.kubernetes.io/last-applied-configuration:
173+
f:spec:
174+
.:
175+
f:resourcePolicy:
176+
.:
177+
f:containerPolicies:
178+
f:targetRef:
179+
.:
180+
f:apiVersion:
181+
f:kind:
182+
f:name:
183+
f:updatePolicy:
184+
.:
185+
f:updateMode:
186+
Manager: kubectl-client-side-apply
187+
Operation: Update
188+
Time: 2022-03-14T12:54:06Z
189+
API Version: autoscaling.k8s.io/v1
190+
Fields Type: FieldsV1
191+
fieldsV1:
192+
f:status:
193+
.:
194+
f:conditions:
195+
f:recommendation:
196+
.:
197+
f:containerRecommendations:
198+
Manager: recommender
199+
Operation: Update
200+
Time: 2022-03-14T12:54:52Z
201+
Resource Version: 3950
202+
UID: 4ac64e4c-c84b-478e-92e4-5f072f985971
203+
Spec:
204+
Resource Policy:
205+
Container Policies:
206+
Container Name: *
207+
Controlled Resources:
208+
cpu
209+
memory
210+
Max Allowed:
211+
Cpu: 1
212+
Memory: 500Mi
213+
Min Allowed:
214+
Cpu: 5m
215+
Memory: 5Mi
216+
Target Ref:
217+
API Version: argoproj.io/v1alpha1
218+
Kind: Rollout
219+
Name: vpa-demo-rollout
220+
Update Policy:
221+
Update Mode: Auto
222+
Status:
223+
Conditions:
224+
Last Transition Time: 2022-03-14T12:54:52Z
225+
Status: True
226+
Type: RecommendationProvided
227+
Recommendation:
228+
Container Recommendations:
229+
Container Name: vpa-demo-rollout
230+
Lower Bound:
231+
Cpu: 25m
232+
Memory: 262144k
233+
Target:
234+
Cpu: 25m
235+
Memory: 262144k
236+
Uncapped Target:
237+
Cpu: 25m
238+
Memory: 262144k
239+
Upper Bound:
240+
Cpu: 1
241+
Memory: 500Mi
242+
Events: <none>
243+
```
244+
245+
Here we see the recommendation for cpu, memory with lowerbound, upper bound, Target etc., are provided. If we check the status of the pods.. the older pods with initial configuration would get terminated and newer pods get created.
246+
247+
```yaml
248+
# kubectl get po -n test-vpa -w
249+
NAME READY STATUS RESTARTS AGE
250+
vpa-demo-rollout-f5df6d577-65f26 1/1 Running 0 17m
251+
vpa-demo-rollout-f5df6d577-d55cx 1/1 Running 0 17m
252+
vpa-demo-rollout-f5df6d577-fdpn2 1/1 Running 0 17m
253+
vpa-demo-rollout-f5df6d577-jg2pw 1/1 Running 0 17m
254+
vpa-demo-rollout-f5df6d577-vlx5x 1/1 Running 0 17m
255+
...
256+
257+
vpa-demo-rollout-f5df6d577-jg2pw 1/1 Terminating 0 17m
258+
vpa-demo-rollout-f5df6d577-vlx5x 1/1 Terminating 0 17m
259+
vpa-demo-rollout-f5df6d577-jg2pw 0/1 Terminating 0 18m
260+
vpa-demo-rollout-f5df6d577-vlx5x 0/1 Terminating 0 18m
261+
vpa-demo-rollout-f5df6d577-w7tx4 0/1 Pending 0 0s
262+
vpa-demo-rollout-f5df6d577-w7tx4 0/1 Pending 0 0s
263+
vpa-demo-rollout-f5df6d577-w7tx4 0/1 ContainerCreating 0 0s
264+
vpa-demo-rollout-f5df6d577-vdlqq 0/1 Pending 0 0s
265+
vpa-demo-rollout-f5df6d577-vdlqq 0/1 Pending 0 1s
266+
vpa-demo-rollout-f5df6d577-jg2pw 0/1 Terminating 0 18m
267+
vpa-demo-rollout-f5df6d577-jg2pw 0/1 Terminating 0 18m
268+
vpa-demo-rollout-f5df6d577-vdlqq 0/1 ContainerCreating 0 1s
269+
vpa-demo-rollout-f5df6d577-w7tx4 1/1 Running 0 6s
270+
vpa-demo-rollout-f5df6d577-vdlqq 1/1 Running 0 7s
271+
vpa-demo-rollout-f5df6d577-vlx5x 0/1 Terminating 0 18m
272+
vpa-demo-rollout-f5df6d577-vlx5x 0/1 Terminating 0 18m
273+
```
274+
275+
If we check the new pod cpu and memory they would be updated as per VPA recommendation:
276+
277+
278+
```yaml
279+
# kubectl describe po vpa-demo-rollout-f5df6d577-vdlqq -n test-vpa
280+
Name: vpa-demo-rollout-f5df6d577-vdlqq
281+
Namespace: test-vpa
282+
Priority: 0
283+
Node: argo-rollouts-control-plane/172.18.0.2
284+
Start Time: Mon, 14 Mar 2022 12:55:06 +0000
285+
Labels: app=vpa-demo-rollout
286+
rollouts-pod-template-hash=f5df6d577
287+
Annotations: vpaObservedContainers: vpa-demo-rollout
288+
vpaUpdates: Pod resources updated by kubengix-vpa: container 0: cpu request, memory request
289+
Status: Running
290+
IP: 10.244.0.17
291+
IPs:
292+
IP: 10.244.0.17
293+
Controlled By: ReplicaSet/vpa-demo-rollout-f5df6d577
294+
Containers:
295+
vpa-demo-rollout:
296+
Container ID: containerd://b79bd88851fe0622d33bc90a1560ca54ef2c27405a3bc9a4fc3a333eef5f9733
297+
Image: ravihari/nginx:v1
298+
Image ID: docker.io/ravihari/nginx@sha256:205961b09a80476af4c2379841bf6abec0022101a7e6c5585a88316f7115d17a
299+
Port: 80/TCP
300+
Host Port: 0/TCP
301+
State: Running
302+
Started: Mon, 14 Mar 2022 12:55:11 +0000
303+
Ready: True
304+
Restart Count: 0
305+
Requests:
306+
cpu: 25m
307+
memory: 262144k
308+
Environment: <none>
309+
Mounts:
310+
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mk4fz (ro)
311+
Conditions:
312+
Type Status
313+
Initialized True
314+
Ready True
315+
ContainersReady True
316+
PodScheduled True
317+
Volumes:
318+
kube-api-access-mk4fz:
319+
Type: Projected (a volume that contains injected data from multiple sources)
320+
TokenExpirationSeconds: 3607
321+
ConfigMapName: kube-root-ca.crt
322+
ConfigMapOptional: <nil>
323+
DownwardAPI: true
324+
QoS Class: Burstable
325+
Node-Selectors: <none>
326+
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
327+
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
328+
Events:
329+
Type Reason Age From Message
330+
---- ------ ---- ---- -------
331+
Normal Scheduled 38s default-scheduler Successfully assigned test-vpa/vpa-demo-rollout-f5df6d577-vdlqq to argo-rollouts-control-plane
332+
Normal Pulled 35s kubelet Container image "ravihari/nginx:v1" already present on machine
333+
Normal Created 35s kubelet Created container vpa-demo-rollout
334+
Normal Started 33s kubelet Started container vpa-demo-rollout
335+
```
336+
337+
## Requirements
338+
In order for the VPA to manipulate the rollout, the Kubernetes cluster hosting the rollout CRD needs the subresources support for CRDs. This feature was introduced as alpha in Kubernetes version 1.10 and transitioned to beta in Kubernetes version 1.11. If a user wants to use VPA on v1.10, the Kubernetes Cluster operator will need to add a custom feature flag to the API server. After 1.10, the flag is turned on by default. Check out the following [link](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) for more information on setting the custom feature flag.
339+
340+
When installing VPA you may need to add the following in RBAC configurations for `system:vpa-target-reader` cluster role as by default VPA maynot support rollouts in all the versions.
341+
342+
```yaml
343+
- apiGroups:
344+
- argoproj.io
345+
resources:
346+
- rollouts
347+
- rollouts/scale
348+
- rollouts/status
349+
- replicasets
350+
verbs:
351+
- get
352+
- list
353+
- watch
354+
```
355+
356+
Makes sure Metrics-Server is installed in the cluster and openssl is upto date for VPA latest version to apply recommendations to the pods properly.

‎mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ nav:
2929
- Canary: features/canary.md
3030
- Rollout Spec: features/specification.md
3131
- HPA: features/hpa-support.md
32+
- VPA: features/vpa-support.md
3233
- Ephemeral Metadata: features/ephemeral-metadata.md
3334
- Restarting Rollouts: features/restart.md
3435
- Scaledown Aborted Rollouts: features/scaledown-aborted-rs.md

0 commit comments

Comments
 (0)