Merge pull request #36102 from abhatt-rh/telcodocs-291-mhc

Bob Furu · web-flow · commit 68ca07ded60e · 2021-10-04T13:41:50.000-04:00
diff --git a/modules/machine-health-checks-about.adoc b/modules/machine-health-checks-about.adoc
@@ -29,9 +29,6 @@ Consider the timeouts carefully, accounting for workloads and requirements.
 
 To stop the check, remove the resource.
 
-For example, you should stop the check during the upgrade process because the nodes in the cluster might become temporarily unavailable. The `MachineHealthCheck` might identify such nodes as unhealthy and reboot them. To avoid rebooting such nodes, remove any `MachineHealthCheck` resource that you have deployed before updating the cluster.
-However, a `MachineHealthCheck` resource that is deployed by default (such as `machine-api-termination-handler`) cannot be removed and will be recreated.
-
 [id="machine-health-checks-limitations_{context}"]
 == Limitations when deploying machine health checks
 
diff --git a/modules/machine-health-checks-pausing.adoc b/modules/machine-health-checks-pausing.adoc
@@ -0,0 +1,69 @@
+// Module included in the following assemblies:
+
+// * updating/updating-cluster-cli.adoc
+// * updating/updating-cluster-between-minor.adoc
+// * updating/updating-restricted-network-cluster.adoc
+
+[id="machine-health-checks-pausing_{context}"]
+= Pausing a MachineHealthCheck resource
+
+During the upgrade process, nodes in the cluster might become temporarily unavailable. In the case of worker nodes, the machine health check might identify such nodes as unhealthy and reboot them. To avoid rebooting such nodes, pause all the `MachineHealthCheck` resources before updating the cluster.
+
+.Prerequisites
+
+* Install the OpenShift CLI (`oc`).
+
+.Procedure
+
+. To list all the available `MachineHealthCheck` resources that you want to pause, run the following command:
++
+[source,terminal]
+----
+$ oc get machinehealthcheck -n openshift-machine-api
+----
+
+. To pause the machine health checks, add the `cluster.x-k8s.io/paused=""` annotation to the `MachineHealthCheck` resource. Run the following command:
++
+[source,terminal]
+----
+$ oc -n openshift-machine-api annotate mhc <mhc-name> cluster.x-k8s.io/paused=""
+----
++
+The annotated `MachineHealthCheck` resource resembles the following YAML file:
++
+[source,yaml]
+----
+apiVersion: machine.openshift.io/v1beta1
+kind: MachineHealthCheck
+metadata:
+  name: example
+  namespace: openshift-machine-api
+  annotations:
+    cluster.x-k8s.io/paused: ""
+spec:
+  selector:
+    matchLabels:
+      role: worker
+  unhealthyConditions:
+  - type:    "Ready"
+    status:  "Unknown"
+    timeout: "300s"
+  - type:    "Ready"
+    status:  "False"
+    timeout: "300s"
+  maxUnhealthy: "40%"
+status:
+  currentHealthy: 5
+  expectedMachines: 5
+----
++
+[IMPORTANT]
+====
+Resume the machine health checks after updating the cluster. To resume the check, remove the pause annotation from the `MachineHealthCheck` resource by running the following command:
+
+[source,terminal]
+----
+$ oc -n openshift-machine-api annotate mhc <mhc-name> cluster.x-k8s.io/paused-
+----
+====
+
diff --git a/modules/update-restricted.adoc b/modules/update-restricted.adoc
@@ -20,6 +20,7 @@ If you have a local OpenShift Update Service, you can update by using the connec
 * You applied the release image signature ConfigMap for the new release to your cluster.
 * You obtained the sha256 sum value for the release from the image signature ConfigMap.
 * Install the OpenShift CLI (`oc`), version 4.4.8 or later.
+* Pause all `MachineHealthCheck` resources.
 
 .Procedure
 
diff --git a/modules/update-service-overview.adoc b/modules/update-service-overview.adoc
@@ -45,8 +45,3 @@ With the specification for the new version applied to the old kubelet, the {op-s
 
 The OpenShift Update Service is composed of an Operator and one or more application instances.
 
-[NOTE]
-====
-During the upgrade process, nodes in the cluster might become temporarily unavailable. The `MachineHealthCheck` might identify such nodes as unhealthy and reboot them. To avoid rebooting such nodes, remove any `MachineHealthCheck` resource that you have deployed before updating the cluster.
-However, a MachineHealthCheck resource that is deployed by default (such as `machine-api-termination-handler`) cannot be removed and will be recreated.
-====
diff --git a/modules/update-upgrading-cli.adoc b/modules/update-upgrading-cli.adoc
@@ -18,6 +18,7 @@ of the Customer Portal.
 * Install the OpenShift CLI (`oc`) that matches the version for your updated version.
 * Log in to the cluster as user with `cluster-admin` privileges.
 * Install the `jq` package.
+* Pause all `MachineHealthCheck` resources.
 
 .Procedure
 
diff --git a/modules/update-upgrading-web.adoc b/modules/update-upgrading-web.adoc
@@ -25,6 +25,7 @@ link:https://access.redhat.com/downloads/content/290[in the errata section] of t
 .Prerequisites
 
 * Have access to the web console as a user with `admin` privileges.
+* Pause all `MachineHealthCheck` resources. 
 
 .Procedure
 
diff --git a/updating/updating-cluster-between-minor.adoc b/updating/updating-cluster-between-minor.adoc
@@ -54,6 +54,8 @@ include::modules/update-using-custom-machine-config-pools-canary.adoc[leveloffse
 
 If you want to use the canary rollout update process, see xref:../updating/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools[Performing a canary rollout update].
 
+include::modules/machine-health-checks-pausing.adoc[leveloffset=+1]
+
 include::modules/update-upgrading-web.adoc[leveloffset=+1]
 
 include::modules/update-changing-update-server-web.adoc[leveloffset=+1]
diff --git a/updating/updating-cluster-cli.adoc b/updating/updating-cluster-cli.adoc
@@ -18,6 +18,7 @@ See xref:../authentication/using-rbac.adoc[Using RBAC to define and apply permis
 * If your cluster uses manually maintained credentials with the AWS Secure Token Service (STS), obtain a copy of the `ccoctl` utility from the release image being upgraded to and use it to process any updated credentials. For more information, see xref:../authentication/managing_cloud_provider_credentials/cco-mode-sts.adoc#sts-mode-upgrading[_Upgrading an OpenShift Container Platform cluster configured for manual mode with STS_].
 * Ensure that you address all `Upgradeable=False` conditions so the cluster allows an upgrade to the next minor version. You can run the `oc adm upgrade` command for an output of all `Upgradeable=False` conditions and the condition reasoning to help you prepare for a minor version upgrade.
 
+
 [IMPORTANT]
 ====
 Using the `unsupportedConfigOverrides` section to modify the configuration of an Operator is unsupported and might block cluster upgrades. You must remove this setting before you can upgrade your cluster.
@@ -35,6 +36,8 @@ include::modules/update-service-overview.adoc[leveloffset=+1]
 
 include::modules/understanding-upgrade-channels.adoc[leveloffset=+1]
 
+include::modules/machine-health-checks-pausing.adoc[leveloffset=+1]
+
 include::modules/update-upgrading-cli.adoc[leveloffset=+1]
 
 include::modules/update-changing-update-server-cli.adoc[leveloffset=+1]
diff --git a/updating/updating-restricted-network-cluster.adoc b/updating/updating-restricted-network-cluster.adoc
@@ -54,6 +54,8 @@ include::modules/update-oc-configmap-signature-verification.adoc[leveloffset=+2]
 
 include::modules/update-configuring-image-signature.adoc[leveloffset=+2]
 
+include::modules/machine-health-checks-pausing.adoc[leveloffset=+1]
+
 include::modules/update-restricted.adoc[leveloffset=+1]
 
 include::modules/images-configuration-registry-mirror.adoc[leveloffset=+1]
