Skip to content

Commit 2895f30

Browse files
mburke5678mburke5678
authored
Merge pull request #39619 from mburke5678/OSDOCS-3047-cgroup2
OSDOCS3047: Dev Preview: Cgroups v2 phase 2
2 parents 0b74cdc + a159e87 commit 2895f30

File tree

6 files changed

+162
-6
lines changed

6 files changed

+162
-6
lines changed

‎modules/installation-configuration-parameters.adoc

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -451,6 +451,10 @@ Optional installation configuration parameters are described in the following ta
451451
|A PEM-encoded X.509 certificate bundle that is added to the nodes' trusted certificate store. This trust bundle may also be used when a proxy has been configured.
452452
|String
453453

454+
|`cgroupsV2`
455+
|Enables link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control groups version 2] (cgroups v2) on specific nodes in your cluster. The {product-title} process for enabling cgroups v2 disables all cgroup version 1 controllers and hierarchies. The {product-title} cgroups version 2 feature is in Developer Preview and is not supported by Red Hat at this time.
456+
|`true`
457+
454458
|`compute`
455459
|The configuration for the machines that comprise the compute nodes.
456460
|Array of `MachinePool` objects.

‎modules/nodes-nodes-cgroups-2.adoc

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/nodes-nodes-working.adoc
4+
// * post_installation_configuration/machine-configuration-tasks.adoc
5+
6+
[id="nodes-nodes-cgroups-2_{context}"]
7+
= Enabling Linux control groups version 2 (cgroups v2)
8+
9+
You can enable link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control groups version 2] (cgroups v2) on specific nodes in your cluster by using a machine config. The {product-title} process for enabling cgroups v2 disables all cgroups version 1 controllers and hierarchies.
10+
11+
[IMPORTANT]
12+
====
13+
The {product-title} cgroups version 2 feature is in Developer Preview and is not supported by Red Hat at this time.
14+
====
15+
16+
.Prerequisites
17+
* You have a running {product-title} cluster that uses version 4.10 or later.
18+
* You are logged in to the cluster as a user with administrative privileges.
19+
* You have the `node-role.kubernetes.io` value for the node(s) you want to configure.
20+
+
21+
[source,terminal]
22+
----
23+
$ oc describe node <node-name>
24+
----
25+
+
26+
.Example output
27+
[source,terminal]
28+
----
29+
Name: ci-ln-v05w5m2-72292-5s9ht-worker-a-r6fpg
30+
Roles: worker
31+
Labels: beta.kubernetes.io/arch=amd64
32+
beta.kubernetes.io/instance-type=n1-standard-4
33+
beta.kubernetes.io/os=linux
34+
failure-domain.beta.kubernetes.io/region=us-central1
35+
failure-domain.beta.kubernetes.io/zone=us-central1-a
36+
kubernetes.io/arch=amd64
37+
kubernetes.io/hostname=ci-ln-v05w5m2-72292-5s9ht-worker-a-r6fpg
38+
kubernetes.io/os=linux
39+
node-role.kubernetes.io/worker= <1>
40+
#...
41+
----
42+
<1> This value is the node role you need.
43+
44+
.Procedure
45+
46+
. Enable cgroups v2 on nodes:
47+
48+
* Create a machine config file YAML, such as `worker-cgroups-v2.yaml`:
49+
+
50+
[source,yaml]
51+
----
52+
apiVersion: machineconfiguration.openshift.io/v1
53+
kind: MachineConfig
54+
metadata:
55+
labels:
56+
machineconfiguration.openshift.io/role: "worker" <1>
57+
name: worker-enable-cgroups-v2
58+
spec:
59+
kernelArguments:
60+
- systemd.unified_cgroup_hierarchy=1 <2>
61+
- cgroup_no_v1="all" <3>
62+
----
63+
<1> Specifies the `node-role.kubernetes.io` value for the nodes you want to configure, such as `master`, `worker`, or `infra`.
64+
<2> Enables cgroups v2 in systemd.
65+
<3> Disables cgroups v1.
66+
67+
* Create the new machine config:
68+
+
69+
[source,terminal]
70+
----
71+
$ oc create -f worker-enable-cgroups-v2.yaml
72+
----
73+
74+
. Check the machine configs to see that the new one was added:
75+
+
76+
[source,terminal]
77+
----
78+
$ oc get MachineConfig
79+
----
80+
+
81+
.Example output
82+
[source,terminal]
83+
----
84+
NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE
85+
00-master 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
86+
00-worker 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
87+
01-master-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
88+
01-master-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
89+
01-worker-container-runtime 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
90+
01-worker-kubelet 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
91+
99-master-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
92+
99-master-ssh 3.2.0 40m
93+
99-worker-generated-registries 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
94+
99-worker-ssh 3.2.0 40m
95+
rendered-master-23e785de7587df95a4b517e0647e5ab7 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
96+
rendered-worker-5d596d9293ca3ea80c896a1191735bb1 52dd3ba6a9a527fc3ab42afac8d12b693534c8c9 3.2.0 33m
97+
worker-enable-cgroups-v2 3.2.0 10s
98+
----
99+
100+
. Check the nodes to see that scheduling on each affected node is disabled. This indicates that the change is being applied:
101+
+
102+
[source,terminal]
103+
----
104+
$ oc get nodes
105+
----
106+
+
107+
.Example output
108+
[source,terminal]
109+
----
110+
NAME STATUS ROLES AGE VERSION
111+
ci-ln-fm1qnwt-72292-99kt6-master-0 Ready master 58m v1.22.1+6859754
112+
ci-ln-fm1qnwt-72292-99kt6-master-1 Ready master 58m v1.22.1+6859754
113+
ci-ln-fm1qnwt-72292-99kt6-master-2 Ready master 58m v1.22.1+6859754
114+
ci-ln-fm1qnwt-72292-99kt6-worker-a-h5gt4 Ready,SchedulingDisabled worker 48m v1.22.1+6859754
115+
ci-ln-fm1qnwt-72292-99kt6-worker-b-7vtmd Ready worker 48m v1.22.1+6859754
116+
ci-ln-fm1qnwt-72292-99kt6-worker-c-rhzkv Ready worker 48m v1.22.1+6859754
117+
----
118+
119+
. After a node returns to the `Ready` state, you can verify that cgroups v2 is enabled by checking that the `sys/fs/cgroup/cgroup.controllers` file is present on the node. This file is created by cgroups v2.
120+
+
121+
* Start a debug session for that node:
122+
+
123+
[source,terminal]
124+
----
125+
$ oc debug node/<node_name>
126+
----
127+
+
128+
* Locate the `sys/fs/cgroup/cgroup.controllers` file. If this file is present, cgroups v2 is enabled on that node.
129+
+
130+
.Example output
131+
[source,terminal]
132+
----
133+
cgroup.controllers cgroup.stat cpuset.cpus.effective io.stat pids
134+
cgroup.max.depth cgroup.subtree_control cpuset.mems.effective kubepods.slice system.slice
135+
cgroup.max.descendants cgroup.threads init.scope memory.pressure user.slice
136+
cgroup.procs cpu.pressure io.pressure memory.stat
137+
----
138+
139+
.Additional resources
140+
141+
* For information about enabling cgroups v2 during installation, see the _Optional parameters_ table in the _Installation configuration parameters_ section of your installation process.
142+

‎modules/nodes-nodes-kernel-arguments.adoc

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,12 @@ Multithreading allows multiple logical threads for each CPU.
2929
You could consider `nosmt` in multi-tenant environments to reduce
3030
risks from potential cross-thread attacks. By disabling SMT, you essentially choose security over performance.
3131

32+
* **systemd.unified_cgroup_hierarchy**: Enables
33+
link:https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control groups version 2] (cgroups v2).
34+
Cgroup v2 is the next version of the kernel
35+
link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01[control groups]
36+
and offers multiple improvements.
37+
3238
See link:https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt[Kernel.org kernel parameters]
3339
for a list and descriptions of kernel arguments.
3440

‎modules/nodes-nodes-working-deleting.adoc

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,5 +60,4 @@ spec:
6060
replicas: 2
6161
----
6262
====
63-
+
64-
For more information on scaling your cluster using a machine set, see _Manually scaling a machine set_.
63+

‎nodes/nodes/nodes-nodes-working.adoc

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11

22
:context: nodes-nodes-working
3+
34
[id="nodes-nodes-working"]
45
= Working with nodes
56
include::modules/common-attributes.adoc[]
@@ -26,14 +27,17 @@ include::modules/nodes-nodes-working-master-schedulable.adoc[leveloffset=+1]
2627
== Deleting nodes
2728

2829
include::modules/nodes-nodes-working-deleting.adoc[leveloffset=+2]
30+
31+
.Additional resources
32+
33+
* For more information on scaling your cluster using a MachineSet,
34+
see xref:../../machine_management/manually-scaling-machineset.adoc#machineset-manually-scaling-manually-scaling-machineset[Manually scaling a MachineSet].
35+
2936
include::modules/nodes-nodes-working-deleting-bare-metal.adoc[leveloffset=+2]
3037

3138
include::modules/nodes-nodes-working-setting-booleans.adoc[leveloffset=+1]
3239
include::modules/nodes-nodes-kernel-arguments.adoc[leveloffset=+1]
3340
ifdef::openshift-webscale[]
3441
include::modules/nodes-nodes-rtkernel-arguments.adoc[leveloffset=+1]
3542
endif::openshift-webscale[]
36-
== Additional resources
37-
38-
For more information on scaling your cluster using a MachineSet,
39-
see xref:../../machine_management/manually-scaling-machineset.adoc#machineset-manually-scaling-manually-scaling-machineset[Manually scaling a MachineSet].
43+
include::modules/nodes-nodes-cgroups-2.adoc[leveloffset=+1]

‎post_installation_configuration/machine-configuration-tasks.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ include::modules/rhcos-enabling-multipath-day-2.adoc[leveloffset=+2]
3535

3636
* See xref:../installing/installing_bare_metal/installing-bare-metal.adoc#rhcos-enabling-multipath_installing-bare-metal[Enabling multipathing with kernel arguments on RHCOS] for more information about enabling multipathing during installation time.
3737

38+
include::modules/nodes-nodes-cgroups-2.adoc[leveloffset=+2]
3839
include::modules/nodes-nodes-rtkernel-arguments.adoc[leveloffset=+2]
3940
include::modules/machineconfig-modify-journald.adoc[leveloffset=+2]
4041
include::modules/rhcos-add-extensions.adoc[leveloffset=+2]

0 commit comments

Comments
 (0)