Skip to content

Commit 5087158

Browse files
committed
Add steps to restore to a previous cluster state
1 parent a1df4c7 commit 5087158

File tree

1 file changed

+51
-0
lines changed

1 file changed

+51
-0
lines changed

modules/dr-restoring-cluster-state.adoc

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,57 @@ etcd-ip-10-0-143-125.ec2.internal 1/1 Running 1
284284
If the status is `Pending`, or the output lists more than one running etcd pod, wait a few minutes and check again.
285285

286286
.. Repeat this step for each lost control plane host that is not the recovery host.
287+
+
288+
[NOTE]
289+
====
290+
Perform the following step only if you are using `OVNKubernetes` Container Network Interface (CNI) plug-in.
291+
====
292+
+
293+
. Restart the Open Virtual Network (OVN) Kubernetes pods on all the hosts.
294+
295+
.. Remove the northbound database (nbdb) and southbound database (sbdb). Access the recovery host and the remaining control plane nodes by using Secure Shell (SSH) and run the following command:
296+
+
297+
[source,terminal]
298+
----
299+
$ sudo rm -f /var/lib/ovn/etc/*.db
300+
----
301+
302+
.. Delete all OVN-Kubernetes control plane pods by running the following command:
303+
+
304+
[source,terminal]
305+
----
306+
$ oc delete pods -l app=ovnkube-master -n openshift-ovn-kubernetes
307+
----
308+
309+
.. Ensure that all the OVN-Kubernetes control plane pods are deployed again and are in a `Running` state by running the following command:
310+
+
311+
[source,terminal]
312+
----
313+
$ oc get pods -l app=ovnkube-master -n openshift-ovn-kubernetes
314+
----
315+
+
316+
.Example output
317+
[source,terminal]
318+
----
319+
NAME READY STATUS RESTARTS AGE
320+
ovnkube-master-nb24h 4/4 Running 0 48s
321+
ovnkube-master-rm8kw 4/4 Running 0 47s
322+
ovnkube-master-zbqnh 4/4 Running 0 56s
323+
----
324+
325+
.. Delete all `ovnkube-node` pods by running the following command:
326+
+
327+
[source,terminal]
328+
----
329+
$ oc get pods -n openshift-ovn-kubernetes -o name | grep ovnkube-node | while read p ; do oc delete $p -n openshift-ovn-kubernetes ; done
330+
----
331+
332+
.. Ensure that all the `ovnkube-node` pods are deployed again and are in a `Running` state by running the following command:
333+
+
334+
[source,terminal]
335+
----
336+
$ oc get pods -n openshift-ovn-kubernetes | grep ovnkube-node
337+
----
287338

288339
. Delete and recreate other non-recovery, control plane machines, one by one. After these machines are recreated, a new revision is forced and etcd scales up automatically.
289340
+

0 commit comments

Comments
 (0)