Skip to content

Commit 01bb9e3

Browse files
jldohmannjldohmann
authored
Merge pull request #70593 from jldohmann/OSDOCS-9369
OSDOCS-9369: add improved mco status updates tech preview
2 parents 72e2c96 + 36ca80c commit 01bb9e3

File tree

2 files changed

+116
-0
lines changed

2 files changed

+116
-0
lines changed

‎modules/checking-mco-node-status.adoc

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * post_installation_configuration/machine-configuration-tasks.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="checking-mco-node-status_{context}"]
7+
= Checking machine config node status
8+
9+
During updates you might want to monitor the progress of individual nodes in case issues arise and you need to troubleshoot a node.
10+
11+
To see the status of the Machine Config Operator (MCO) updates to your cluster, use the following `oc` commands:
12+
13+
:FeatureName: Improved MCO state reporting
14+
include::snippets/technology-preview.adoc[]
15+
16+
.Procedure
17+
. Get a summary of update statuses for all nodes in all machine config pools by running the following command:
18+
+
19+
[source,terminal]
20+
----
21+
$ oc get machineconfignodes
22+
----
23+
+
24+
.Example output
25+
[source,text]
26+
----
27+
NAME UPDATED UPDATEPREPARED UPDATEEXECUTED UPDATEPOSTACTIONCOMPLETED UPDATECOMPLETED RESUMED
28+
ip-10-0-12-194.ec2.internal True False False False False False
29+
ip-10-0-17-102.ec2.internal False True False False False False
30+
ip-10-0-2-232.ec2.internal False False True False False False
31+
ip-10-0-59-251.ec2.internal False False False True False False
32+
ip-10-0-59-56.ec2.internal False False False False True True
33+
ip-10-0-6-214.ec2.internal False False Unknown False False False
34+
----
35+
+
36+
where:
37+
+
38+
UPDATED:: The `True` status indicates that the MCO has applied the current machine config to that particular node. The `False` status indicates that the node is currently updating. The `Unknown` status means the operation is processing.
39+
UPDATEPREPARED:: The `False` status indicates that the MCO has not started reconciling the new machine configs to be distributed. The `True` status indicates that the MCO has completed this phase of the update. The `Unknown` status means the operation is processing.
40+
UPDATEEXECUTED:: The `False` status indicates that the MCO has not started cordoning and draining the node. It also indicates that the disk state and operating system have not started updating. The `True` status indicates that the MCO has completed this phase of the update. The `Unknown` status means the operation is processing.
41+
UPDATEPOSTACTIONCOMPLETED:: The `False` status indicates that the MCO has not started rebooting the node or closing the daemon. The `True` status indicates that the MCO has completed reboot and updating the node status. The `Unknown` status indicates either that an error has occurred during the update process at this phase, or that the MCO is currently applying the update.
42+
UPDATECOMPLETED:: The `False` status indicates that the MCO has not started uncordoning the node and updating the node state and metrics. The `True` status indicates that the MCO has finished updating the node state and available metrics.
43+
RESUMED:: The `False` status indicates that the MCO has not started the config drift monitor. The `True` status indicates that the node has resumed operation. The `Unknown` status means the operation is processing.
44+
+
45+
[NOTE]
46+
====
47+
Within the primary phases previously described, you can have secondary phases which you can use to see the update progression in more detail. You can get more information that includes secondary phases of updates by using the `-o wide` option of the preceding command. This provides the additional `UPDATECOMPATIBLE`, `UPDATEFILESANDOS`, `DRAINEDNODE`, `CORDONEDNODE`, `REBOOTNODE`, `RELOADEDCRIO` and `UNCORDONED` columns. These secondary phases do not always occur and depend on the type of update you want to apply.
48+
====
49+
50+
. Check the update status of nodes in a specific machine config pool by running the following command:
51+
+
52+
[source,terminal]
53+
----
54+
$ oc get machineconfignodes $(oc get machineconfignodes -o json | jq -r '.items[]|select(.spec.pool.name=="<pool_name>")|.metadata.name') <1>
55+
----
56+
<1> The name of the pool is the `MachineConfigPool` object name.
57+
+
58+
.Example output
59+
[source,text]
60+
----
61+
NAME UPDATED UPDATEPREPARED UPDATEEXECUTED UPDATEPOSTACTIONCOMPLETE UPDATECOMPLETE RESUMED
62+
ip-10-0-48-226.ec2.internal True False False False False False
63+
ip-10-0-5-241.ec2.internal True False False False False False
64+
ip-10-0-74-108.ec2.internal True False False False False False
65+
----
66+
67+
. Check the update status of an individual node by running the following command:
68+
+
69+
[source,terminal]
70+
----
71+
$ oc describe machineconfignode/<node_name> <1>
72+
----
73+
<1> The name of the node is the `MachineConfigNode` object name.
74+
+
75+
.Example output
76+
[source,text]
77+
----
78+
Name: <node_name>
79+
Namespace:
80+
Labels: <none>
81+
Annotations: <none>
82+
API Version: machineconfiguration.openshift.io/v1alpha1
83+
Kind: MachineConfigNode
84+
Metadata:
85+
Creation Timestamp: 2023-10-17T13:08:58Z
86+
Generation: 1
87+
Resource Version: 49443
88+
UID: 4bd758ab-2187-413c-ac42-882e61761b1d
89+
Spec:
90+
Node Ref:
91+
Name: <node_name>
92+
Pool:
93+
Name: master
94+
ConfigVersion:
95+
Desired: rendered-worker-823ff8dc2b33bf444709ed7cd2b9855b <1>
96+
Status:
97+
Conditions:
98+
Last Transition Time: 2023-10-17T13:09:02Z
99+
Message: Node has completed update to config rendered-master-cf99e619747ab19165f11e3546c71f1e
100+
Reason: NodeUpgradeComplete
101+
Status: True
102+
Type: Updated
103+
Last Transition Time: 2023-10-17T13:09:02Z
104+
Message: This node has not yet entered the UpdatePreparing phase
105+
Reason: NotYetOccured
106+
Status: False
107+
Config Version:
108+
Current: rendered-worker-823ff8dc2b33bf444709ed7cd2b9855b
109+
Desired: rendered-worker-823ff8dc2b33bf444709ed7cd2b9855b <2>
110+
Health: Healthy
111+
Most Recent Error:
112+
Observed Generation: 3
113+
----
114+
<1> The desired configuration specified in the `spec.configversion.desired` field updates immediately when a new configuration is detected on the node.
115+
<2> The desired configuration specified in the `status.configversion.desired` field updates only when the new configuration is validated by the Machine Config Daemon (MCD). The MCD performs validation by checking the current phase of the update. If the update successfully passes the `UPDATEPREPARED` phase, then the status adds the new configuration.

‎post_installation_configuration/machine-configuration-tasks.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ include::modules/machine-config-operator.adoc[leveloffset=+2]
2626
include::modules/machine-config-overview.adoc[leveloffset=+2]
2727
include::modules/machine-config-drift-detection.adoc[leveloffset=+2]
2828
include::modules/checking-mco-status.adoc[leveloffset=+2]
29+
include::modules/checking-mco-node-status.adoc[leveloffset=+2]
2930
include::modules/checking-mco-status-certs.adoc[leveloffset=+2]
3031

3132
[id="using-machineconfigs-to-change-machines"]

0 commit comments

Comments
 (0)