Skip to content

Commit af257f4

Browse files
mburke5678mburke5678
authored
Merge pull request #53696 from mburke5678/okd-cgroup-v2-by-default
GH53536: [OKD] Documentation is unclear about status of cgroups v2
2 parents 157b72a + f5ce270 commit af257f4

File tree

6 files changed

+214
-2
lines changed

6 files changed

+214
-2
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -489,6 +489,7 @@ Topics:
489489
File: configuring-firewall
490490
- Name: Enabling Linux control group version 2 (cgroup v2)
491491
File: enabling-cgroup-v2
492+
Distros: openshift-enterprise
492493
- Name: Validating an installation
493494
File: validating-an-installation
494495
Distros: openshift-origin,openshift-enterprise
@@ -2160,7 +2161,10 @@ Topics:
21602161
Distros: openshift-enterprise,openshift-origin
21612162
- Name: Enabling Linux control group version 2 (cgroup v2)
21622163
File: nodes-cluster-cgroups-2
2163-
Distros: openshift-enterprise,openshift-origin
2164+
Distros: openshift-enterprise
2165+
- Name: Configuring the Linux cgroup on your nodes
2166+
File: nodes-cluster-cgroups-okd
2167+
Distros: openshift-origin
21642168
- Name: Enabling features using FeatureGates
21652169
File: nodes-cluster-enabling-features
21662170
Distros: openshift-enterprise,openshift-origin

‎modules/nodes-clusters-cgroups-okd-configure.adoc

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/clusters/nodes-cluster-cgroups-okd.adoc
4+
// * post_installation_configuration/cluster-tasks.adoc
5+
6+
ifeval::["{context}" == "nodes-cluster-cgroups-2"]
7+
:node:
8+
endif::[]
9+
ifeval::["{context}" == "post-install-cluster-tasks"]
10+
:post:
11+
endif::[]
12+
13+
:_content-type: PROCEDURE
14+
[id="nodes-clusters-cgroups-okd-configure_{context}"]
15+
= Configuring Linux cgroup
16+
17+
ifdef::post[]
18+
By default, {product-title} uses link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control group version 2] (cgroup v2) in your cluster. You can switch to link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1.html[Linux control group version 1] (cgroup v1), if needed.
19+
20+
cgroup v2 is the next version of the kernel link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01[control group] and offers multiple improvements. However, it can have some unwanted effects on your nodes.
21+
endif::post[]
22+
23+
You can configure whether your cluster uses cgroup v1 or cgroup v2 by editing the `node.config` object. Enabling the other version of cgroup in {product-title} disables the current cgroup controllers and hierarchies in your cluster.
24+
25+
.Prerequisites
26+
* Have administrative privilege to a working {product-title} cluster.
27+
28+
.Procedure
29+
30+
. Create a `MachineConfig` object file that identifies the kernel argument (for example, `worker-cgroup-v1.yaml`)
31+
+
32+
[source,yaml]
33+
----
34+
apiVersion: machineconfiguration.openshift.io/v1
35+
kind: MachineConfig
36+
metadata:
37+
labels:
38+
machineconfiguration.openshift.io/role: worker <1>
39+
name: worker-cgroup-v1 <2>
40+
spec:
41+
config:
42+
ignition:
43+
version: 3.2.0
44+
kernelArguments:
45+
- systemd.unified_cgroup_hierarchy=0 <3>
46+
----
47+
+
48+
<1> Applies the new kernel argument only to worker nodes.
49+
<2> Applies a name to the machine config.
50+
<3> Configures cgroup v1 on the associated nodes.
51+
52+
. Create the new machine config:
53+
+
54+
[source,terminal]
55+
----
56+
$ oc create -f 05-worker-cgroup-v1.yaml
57+
----
58+
59+
. Check to see that the new machine config was added:
60+
+
61+
[source,terminal]
62+
----
63+
$ oc get MachineConfig
64+
----
65+
+
66+
.Example output
67+
[source,terminal]
68+
----
69+
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
70+
00-master 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
71+
00-worker 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
72+
01-master-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
73+
01-master-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
74+
01-worker-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
75+
01-worker-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
76+
99-worker-cgroup-v1 3.2.0 105s
77+
99-master-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
78+
99-master-ssh 3.2.0 40m
79+
99-worker-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
80+
99-worker-ssh 3.2.0 40m
81+
rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
82+
rendered-master-c5e92d98103061c4818cfcefcf462770 60746a843e7ef8855ae00f2ffcb655c53e0e8296 3.2.0 115s
83+
rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
84+
----
85+
86+
. Check the nodes:
87+
+
88+
[source,terminal]
89+
----
90+
$ oc get nodes
91+
----
92+
+
93+
.Example output
94+
[source,terminal]
95+
----
96+
NAME STATUS ROLES AGE VERSION
97+
ip-10-0-136-161.ec2.internal Ready worker 28m v1.25.0
98+
ip-10-0-136-243.ec2.internal Ready master 34m v1.25.0
99+
ip-10-0-141-105.ec2.internal Ready,SchedulingDisabled worker 28m v1.25.0
100+
ip-10-0-142-249.ec2.internal Ready master 34m v1.25.0
101+
ip-10-0-153-11.ec2.internal Ready worker 28m v1.25.0
102+
ip-10-0-153-150.ec2.internal Ready master 34m v1.25.0
103+
----
104+
+
105+
You can see that The command disables scheduling on each worker node.
106+
107+
. Check that the sys/fs/cgroup/cgroup2fs file has been moved to the `tmpfs` file system:
108+
+
109+
[source,terminal]
110+
----
111+
$ stat -c %T -f /sys/fs/cgroup
112+
----
113+
+
114+
.Example output
115+
+
116+
[source,terminal]
117+
----
118+
tmpfs
119+
----
120+
121+
ifeval::["{context}" == "nodes-cluster-cgroups-2"]
122+
:!node:
123+
endif::[]
124+
ifeval::["{context}" == "post-install-cluster-tasks"]
125+
:!post:
126+
endif::[]

‎modules/nodes-nodes-kernel-arguments.adoc

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,30 @@ Examples of kernel arguments you could set include:
2020
2121
* **nosmt**: Disables symmetric multithreading (SMT) in the kernel. Multithreading allows multiple logical threads for each CPU. You could consider `nosmt` in multi-tenant environments to reduce risks from potential cross-thread attacks. By disabling SMT, you essentially choose security over performance.
2222
23+
ifndef::openshift-origin[]
2324
* **systemd.unified_cgroup_hierarchy**: Enables link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control group version 2] (cgroup v2). cgroup v2 is the next version of the kernel link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01[control group] and offers multiple improvements.
25+
+
26+
[IMPORTANT]
27+
====
28+
[subs="attributes+"]
29+
{product-title} cgroups version 2 support is a Technology Preview feature only. Technology Preview features
30+
are not supported with Red Hat production service level agreements (SLAs) and
31+
might not be functionally complete. Red Hat does not support Technology Preview features for use in production systems. These features provide early access to upcoming product
32+
features, enabling customers to test functionality and provide feedback during
33+
the development process.
34+
35+
For more information about the support scope of Red Hat Technology Preview
36+
features, see https://access.redhat.com/support/offerings/techpreview/.
37+
====
38+
endif::openshift-origin[]
39+
ifdef::openshift-origin[]
40+
* **systemd.unified_cgroup_hierarchy**: Configures the version of Linux control group that is installed on your nodes: link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1.html[cgroup v1] or link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[cgroup v2]. cgroup v2 is the next version of the kernel link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01[control group] and offers multiple improvements. However, it can have some unwanted effects on your nodes.
41+
+
42+
[NOTE]
43+
====
44+
cgroup v2 is enabled by default. To disable cgroup v2, use the `systemd.unified_cgroup_hierarchy=0` kernel argument, as shown in the following procedure.
45+
====
46+
endif::openshift-origin[]
2447

2548
See link:https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt[Kernel.org kernel parameters] for a list and descriptions of kernel arguments.
2649

@@ -61,6 +84,7 @@ rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12
6184
rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
6285
----
6386

87+
ifndef::openshift-origin[]
6488
. Create a `MachineConfig` object file that identifies the kernel argument (for example, `05-worker-kernelarg-selinuxpermissive.yaml`)
6589
+
6690
[source,yaml]
@@ -83,13 +107,46 @@ spec:
83107
<2> Named to identify where it fits among the machine configs (05) and what it does (adds
84108
a kernel argument to configure SELinux permissive mode).
85109
<3> Identifies the exact kernel argument as `enforcing=0`.
110+
. Create the new machine config:
111+
+
112+
[source,terminal]
113+
----
114+
$ oc create -f 05-worker-kernelarg-selinuxpermissive.yaml
115+
----
116+
endif::openshift-origin[]
117+
ifdef::openshift-origin[]
118+
. Create a `MachineConfig` object file that identifies the kernel argument (for example, `05-worker-kernelarg-selinuxpermissive.yaml`)
119+
+
120+
[source,yaml]
121+
----
122+
apiVersion: machineconfiguration.openshift.io/v1
123+
kind: MachineConfig
124+
metadata:
125+
labels:
126+
machineconfiguration.openshift.io/role: worker <1>
127+
name: 05-worker-kernelarg-selinuxpermissive <2>
128+
spec:
129+
config:
130+
ignition:
131+
version: 3.2.0
132+
kernelArguments:
133+
- enforcing=0 <3>
134+
systemd.unified_cgroup_hierarchy=0 <4>
135+
----
136+
+
137+
<1> Applies the new kernel argument only to worker nodes.
138+
<2> Named to identify where it fits among the machine configs (05) and what it does (adds
139+
a kernel argument to configure SELinux permissive mode).
140+
<3> Identifies the exact kernel argument as `enforcing=0`.
141+
<4> Configures cgroup v1 on the associated nodes. cgroup v2 is the default.
86142

87143
. Create the new machine config:
88144
+
89145
[source,terminal]
90146
----
91147
$ oc create -f 05-worker-kernelarg-selinuxpermissive.yaml
92148
----
149+
endif::openshift-origin[]
93150

94151
. Check the machine configs to see that the new one was added:
95152
+

‎nodes/clusters/nodes-cluster-cgroups-okd.adoc

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
:_content-type: ASSEMBLY
2+
:context: nodes-cluster-cgroups-2
3+
[id="nodes-cluster-cgroups-okd"]
4+
= Configuring the Linux cgroup on your nodes
5+
include::_attributes/common-attributes.adoc[]
6+
7+
toc::[]
8+
9+
By default, {product-title} uses link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control group version 2] (cgroup v2) in your cluster. You can switch to link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1.html[Linux control group version 1] (cgroup v1), if needed.
10+
11+
cgroup v2 is the next version of the kernel link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01[control group] and offers multiple improvements. However, it can have some unwanted effects on your nodes.
12+
13+
// The following include statements pull in the module files that comprise
14+
// the assembly. Include any combination of concept, procedure, or reference
15+
// modules required to cover the user story. You can also include other
16+
// assemblies.
17+
18+
include::modules/nodes-clusters-cgroups-okd-configure.adoc[leveloffset=+1]
19+

‎post_installation_configuration/cluster-tasks.adoc

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -587,8 +587,12 @@ include::modules/machine-autoscaler-cr.adoc[leveloffset=+2]
587587
:FeatureResourceName: MachineAutoscaler
588588
include::modules/deploying-resource.adoc[leveloffset=+2]
589589

590+
ifndef::openshift-origin[]
590591
include::modules/nodes-clusters-cgroups-2.adoc[leveloffset=+1]
591-
592+
endif::openshift-origin[]
593+
ifdef::openshift-origin[]
594+
include::modules/nodes-clusters-cgroups-okd-configure.adoc[leveloffset=+1]
595+
endif::openshift-origin[]
592596
[id="post-install-tp-tasks"]
593597
== Enabling Technology Preview features using FeatureGates
594598

‎post_installation_configuration/machine-configuration-tasks.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,3 +74,5 @@ Besides managing `MachineConfig` objects, the MCO manages two custom resources (
7474
include::modules/create-a-kubeletconfig-crd-to-edit-kubelet-parameters.adoc[leveloffset=+2]
7575
include::modules/create-a-containerruntimeconfig-crd.adoc[leveloffset=+2]
7676
include::modules/set-the-default-max-container-root-partition-size-for-overlay-with-crio.adoc[leveloffset=+2]
77+
78+

0 commit comments

Comments
 (0)