Merge pull request #90359 from openshift-cherrypick-robot/cherry-pick-90079-to-enterprise-4.16

[enterprise-4.16] [OCPBUGS-49434]: Revising etcd restore procedure

Author: lahinson

modules/dr-restoring-cluster-state.adoc

@@ -413,6 +413,77 @@ $ oc -n openshift-ovn-kubernetes delete pod -l app=ovnkube-node --field-selector
 ----
 +
 
+.. Check the status of the OVN pods by running the following command:
++
+[source,terminal]
+----
+$ oc get po -n openshift-ovn-kubernetes
+----
++
+
+... If any OVN pods are in the `Terminating` status, delete the node that is running that OVN pod by running the following command. Replace `<node>` with the name of the node you are deleting:
++
+[source,terminal]
+----
+$ oc delete node <node>
+----
++
+
+... Use SSH to log in to the OVN pod node with the `Terminating` status by running the following command:
++
+[source,terminal]
+----
+$ ssh -i <ssh-key-path> core@<node>
+----
++
+
+... Move all PEM files from the `/var/lib/kubelet/pki` directory by running the following command:
++
+[source,terminal]
+----
+$ sudo mv /var/lib/kubelet/pki/* /tmp
+----
++
+
+... Restart the kubelet service by running the following command:
++
+[source,terminal]
+----
+$ sudo systemctl restart kubelet.service
+----
++
+
+... Return to the recovery etcd machines by running the following command:
++
+[source,terminal]
+----
+$ oc get csr
+----
++
+.Example output
++
+[source,terminal]
+----
+NAME        AGE    SIGNERNAME                         REQUESTOR                     CONDITION                                                            
+csr-<uuid>   8m3s   kubernetes.io/kubelet-serving     system:node:<node_name>       Pending  
+----
+
+... Approve all new CSRs by running the following command, replacing `csr-<uuid>` with the name of the CSR:
++
+[source,terminal]
+----
+oc adm certificate approve csr-<uuid>
+----
++
+
+... Verify that the node is back by running the following command:
++
+[source,terminal]
+----
+$ oc get nodes
+----
++
+
 .. Verify that the `ovnkube-node` pod is running again with:
 +
 [source,terminal]