Merge pull request #60860 from bergerhoffer/OSDOCS-3366

OSDOCS-3366: Adding steps to check kubelet cert expirations

Author: bergerhoffer

modules/graceful-shutdown.adoc

@@ -33,18 +33,68 @@ If your cluster fails to recover, follow the steps to restore to a previous clus
 
 .Procedure
 
-. If you are shutting the cluster down for an extended period, determine the date on which certificates expire.
+. If you plan to shut down the cluster for an extended period of time, determine the date that cluster certificates expire.
++
+You must restart the cluster prior to the date that certificates expire. As the cluster restarts, the process might require you to manually approve the pending certificate signing requests (CSRs) to recover kubelet certificates.
+
+.. Check the expiration date for the `kube-apiserver-to-kubelet-signer` CA certificate:
++
+[source,terminal]
+----
+$ oc -n openshift-kube-apiserver-operator get secret kube-apiserver-to-kubelet-signer -o jsonpath='{.metadata.annotations.auth\.openshift\.io/certificate-not-after}{"\n"}'
+----
++
+.Example output
+[source,terminal]
+----
+2023-08-05T14:37:50Z
+----
+
+.. Check the expiration date for the kubelet certificates:
+
+... Start a debug session for a control plane node by running the following command:
++
+[source,terminal]
+----
+$ oc debug node/<node_name>
+----
+
+... Change your root directory to `/host` by running the following command:
 +
 [source,terminal]
 ----
-$ oc -n openshift-kube-apiserver-operator get secret kube-apiserver-to-kubelet-signer -o jsonpath='{.metadata.annotations.auth\.openshift\.io/certificate-not-after}'
+sh-4.4# chroot /host
+----
+
+... Check the kubelet client certificate expiration date by running the following command:
++
+[source,terminal]
+----
+sh-5.1# openssl x509 -in /var/lib/kubelet/pki/kubelet-client-current.pem -noout -enddate
+----
++
+.Example output
+[source,terminal]
+----
+notAfter=Jun  6 10:50:07 2023 GMT
+----
+
+... Check the kubelet server certificate expiration date by running the following command:
++
+[source,terminal]
+----
+sh-5.1# openssl x509 -in /var/lib/kubelet/pki/kubelet-server-current.pem -noout -enddate
 ----
 +
 .Example output
+[source,terminal]
 ----
-2022-08-05T14:37:50Zuser@user:~ $ <1>
+notAfter=Jun  6 10:50:07 2023 GMT
 ----
-<1> To ensure that the cluster can restart gracefully, plan to restart it on or before the specified date. As the cluster restarts, the process might require you to manually approve the pending certificate signing requests (CSRs) to recover kubelet certificates.
+
+... Exit the debug session.
+
+... Repeat these steps to check certificate expiration dates on all control plane nodes. To ensure that the cluster can restart gracefully, plan to restart it before the earliest certificate expiration date.
 
 . Shut down all of the nodes in the cluster. You can do this from your cloud provider's web console, or run the following loop:
 +