Merge pull request #45004 from mburke5678/BZ-2078674

BZ2078674: Restoring cluster to a previous state fail in case nodes certificate are updated after the last backup

Author: mburke5678

modules/dr-restoring-cluster-state.adoc

@@ -139,6 +139,53 @@ static-pod-resources/kube-controller-manager-pod-7/kube-controller-manager-pod.y
 starting kube-scheduler-pod.yaml
 static-pod-resources/kube-scheduler-pod-8/kube-scheduler-pod.yaml
 ----
++
+[NOTE]
+====
+The restore process can cause nodes to enter the `NotReady` state if the node certificates were updated after the last etcd backup. 
+====
+
+. Check the nodes to ensure they are in the `Ready` state.
+
+.. Run the following command:
++
+[source,terminal]
+----
+$ oc get nodes -w
+----
++
+.Sample output
+[source,terminal]
+----
+NAME                STATUS  ROLES          AGE     VERSION
+host-172-25-75-28   Ready   master         3d20h   v1.23.3+e419edf
+host-172-25-75-38   Ready   infra,worker   3d20h   v1.23.3+e419edf
+host-172-25-75-40   Ready   master         3d20h   v1.23.3+e419edf
+host-172-25-75-65   Ready   master         3d20h   v1.23.3+e419edf
+host-172-25-75-74   Ready   infra,worker   3d20h   v1.23.3+e419edf
+host-172-25-75-79   Ready   worker         3d20h   v1.23.3+e419edf
+host-172-25-75-86   Ready   worker         3d20h   v1.23.3+e419edf
+host-172-25-75-98   Ready   infra,worker   3d20h   v1.23.3+e419edf
+----
++
+It can take several minutes for all nodes to report their state. 
+
+.. If any nodes are in the `NotReady` state, log in to the nodes and remove all of the PEM files from the `/var/lib/kubelet/pki` directory on each node. You can SSH into the nodes or use the terminal window in the web console.
++
+[source,terminal]
+----
+$  ssh -i <ssh-key-path> core@<master-hostname>
+----
++
+.Sample `pki` directory
+[sample,terminal]
+----
+sh-4.4# pwd
+/var/lib/kubelet/pki
+sh-4.4# ls
+kubelet-client-2022-04-28-11-24-09.pem  kubelet-server-2022-04-28-11-24-15.pem
+kubelet-client-current.pem              kubelet-server-current.pem
+----
 
 . Restart the kubelet service on all control plane hosts.