Skip to content

Commit d5bb208

Browse files
opayne1opayne1
authored
Merge pull request #63875 from lahinson/osdocs-7196-etcd-tuning-parameters
[OSDOCS-7196]: Adding TP docs for etcd tuning parameters
2 parents ac3b673 + db5f8f9 commit d5bb208

File tree

2 files changed

+146
-0
lines changed

2 files changed

+146
-0
lines changed

‎modules/etcd-tuning-parameters.adoc

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * scalability_and_performance/recommended-performance-scale-practices/recommended-etcd-practices.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="etcd-tuning-parameters_{context}"]
7+
= Setting tuning parameters for etcd
8+
9+
You can set the control plane hardware speed to `"Standard"`, `"Slower"`, or the default, which is `""`.
10+
11+
The default setting allows the system to decide which speed to use. This value enables upgrades from versions where this feature does not exist, as the system can select values from previous versions.
12+
13+
By selecting one of the other values, you are overriding the default. If you see many leader elections due to timeouts or missed heartbeats and your system is set to `""` or `"Standard"`, set the hardware speed to `"Slower"` to make the system more tolerant to the increased latency.
14+
15+
:FeatureName: Tuning etcd latency tolerances
16+
include::snippets/technology-preview.adoc[]
17+
18+
[id="etcd-changing-hardware-speed-tolerance_{context}"]
19+
== Changing hardware speed tolerance
20+
21+
To change the hardware speed tolerance for etcd, complete the following steps.
22+
23+
.Procedure
24+
25+
. Check to see what the current value is by entering the following command:
26+
+
27+
[source, terminal]
28+
----
29+
$ oc describe etcd/cluster | grep "Control Plane Hardware Speed"
30+
----
31+
+
32+
.Example output
33+
[source,terminal]
34+
----
35+
Control Plane Hardware Speed: <VALUE>
36+
----
37+
+
38+
[NOTE]
39+
====
40+
If the output is empty, the field has not been set and should be considered as the default ("").
41+
====
42+
43+
. Change the value by entering the following command. Replace `<value>` with one of the valid values: `""`, `"Standard"`, or `"Slower"`:
44+
+
45+
[source,terminal]
46+
----
47+
oc patch etcd/cluster --type=merge -p '{"spec": {"controlPlaneHardwareSpeed": "<value>"}}'
48+
----
49+
+
50+
The following table indicates the heartbeat interval and leader election timeout for each profile. These values are subject to change.
51+
+
52+
|===
53+
| Profile | ETCD_HEARTBEAT_INTERVAL | ETCD_LEADER_ELECTION_TIMEOUT
54+
| `""` | Varies depending on platform | Varies depending on platform
55+
| `Standard` | 100 | 1000
56+
| `Slower` | 500 | 2500
57+
|===
58+
59+
. Review the output:
60+
+
61+
.Example output
62+
[source,terminal]
63+
----
64+
etcd.operator.openshift.io/cluster patched
65+
----
66+
+
67+
If you enter any value besides the valid values, error output is displayed. For example, if you entered `"Faster"` as the value, the output is as follows:
68+
+
69+
.Example output
70+
[source,terminal]
71+
----
72+
The Etcd "cluster" is invalid: spec.controlPlaneHardwareSpeed: Unsupported value: "Faster": supported values: "", "Standard", "Slower"
73+
----
74+
75+
. Verify that the value was changed by entering the following command:
76+
+
77+
[source, terminal]
78+
----
79+
$ oc describe etcd/cluster | grep "Control Plane Hardware Speed"
80+
----
81+
+
82+
.Example output
83+
[source,terminal]
84+
----
85+
Control Plane Hardware Speed: ""
86+
----
87+
88+
. Force an etcd rollout, which restarts all etcd pods one at a time to pick up the new values, by entering the following command:
89+
+
90+
[source,terminal]
91+
----
92+
$ oc patch etcd/cluster --type=merge -p '{"spec": {"forceRedeploymentReason": "hardwareSpeedChange-'"$( date --rfc-3339=ns )"'"}}'
93+
----
94+
+
95+
.Example output
96+
[source,terminal]
97+
----
98+
etcd.operator.openshift.io/cluster patched
99+
----
100+
101+
. Wait for etcd pods to roll out:
102+
+
103+
[source,terminal]
104+
----
105+
oc get pods -n openshift-etcd -w
106+
----
107+
+
108+
The following output shows the expected entries for master-0. Before you continue, wait until all masters show a status of `4/4 Running`.
109+
+
110+
.Example output
111+
[source,terminal]
112+
----
113+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 Pending 0 0s
114+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 Pending 0 0s
115+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 ContainerCreating 0 0s
116+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 ContainerCreating 0 1s
117+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 1/1 Running 0 2s
118+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 Completed 0 34s
119+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 Completed 0 36s
120+
installer-9-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 Completed 0 36s
121+
etcd-guard-ci-ln-qkgs94t-72292-9clnd-master-0 0/1 Running 0 26m
122+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 4/4 Terminating 0 11m
123+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 4/4 Terminating 0 11m
124+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 0/4 Pending 0 0s
125+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 0/4 Init:1/3 0 1s
126+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 0/4 Init:2/3 0 2s
127+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 0/4 PodInitializing 0 3s
128+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 3/4 Running 0 4s
129+
etcd-guard-ci-ln-qkgs94t-72292-9clnd-master-0 1/1 Running 0 26m
130+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 3/4 Running 0 20s
131+
etcd-ci-ln-qkgs94t-72292-9clnd-master-0 4/4 Running 0 20s
132+
----
133+
134+
. Enter the following command to review to the values:
135+
+
136+
[source,terminal]
137+
----
138+
$ oc describe -n openshift-etcd pod/<ETCD_PODNAME> | grep -e HEARTBEAT_INTERVAL -e ELECTION_TIMEOUT
139+
----
140+
+
141+
[NOTE]
142+
====
143+
These values might not have changed from the default.
144+
====

‎scalability_and_performance/recommended-performance-scale-practices/recommended-etcd-practices.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,5 @@ include::modules/move-etcd-different-disk.adoc[leveloffset=+1]
2222
* link:https://docs.openshift.com/container-platform/4.11/architecture/architecture-rhcos.html[Red Hat Enterprise Linux CoreOS (RHCOS)]
2323
2424
include::modules/etcd-defrag.adoc[leveloffset=+1]
25+
26+
include::modules/etcd-tuning-parameters.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)