Skip to content

Commit 94d0c37

Browse files
sjhala-ccssjhala-ccs
authored
Merge pull request #67868 from sjhala-ccs/oadp-2823
OADP#2823: Added OADP 1.3 Data Mover section
2 parents 0efce01 + bfbde82 commit 94d0c37

File tree

5 files changed

+286
-0
lines changed

5 files changed

+286
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2849,6 +2849,13 @@ Topics:
28492849
File: oadp-12-data-mover-ceph-doc
28502850
- Name: Cleaning up after a backup using OADP 1.1 Data Mover
28512851
File: oadp-cleaning-up-after-data-mover-1-1-backup-doc
2852+
- Name: OADP 1.3 Data Mover
2853+
Dir: installing
2854+
Topics:
2855+
- Name: About the OADP 1.3 Data Mover
2856+
File: about-oadp-1-3-data-mover
2857+
- Name: Backing up and restoring volumes by using CSI snapshots
2858+
File: oadp-backup-restore-csi-snapshots
28522859
- Name: Troubleshooting
28532860
File: troubleshooting
28542861
- Name: OADP API

‎backup_and_restore/application_backup_and_restore/installing/about-oadp-1-3-data-mover.adoc

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="about-oadp-1-3-data-mover"]
3+
= About the OADP 1.3 Data Mover
4+
include::_attributes/common-attributes.adoc[]
5+
:context: about-oadp-1-3-data-mover
6+
7+
toc::[]
8+
9+
OADP 1.3 includes a built-in Data Mover that you can use to move Container Storage Interface (CSI) volume snapshots to a remote object store. The built-in Data Mover allows you to restore stateful applications from the remote object store if a failure, accidental deletion, or corruption of the cluster occurs. It uses xref:../../../backup_and_restore/application_backup_and_restore/backing_up_and_restoring/oadp-about-kopia.adoc#oadp-about-kopia[Kopia] as the uploader mechanism to read the snapshot data and write to the unified repository.
10+
11+
OADP supports CSI snapshots on the following:
12+
13+
* Red Hat OpenShift Data Foundation
14+
* Any other cloud storage provider with the Container Storage Interface (CSI) driver that supports the Kubernetes Volume Snapshot API
15+
16+
:FeatureName: The OADP built-in Data Mover
17+
include::snippets/technology-preview.adoc[]
18+
19+
[id="enabling-oadp-1-3-data-mover"]
20+
== Enabling the built-in Data Mover
21+
22+
To enable the built-in Data Mover, you must include the CSI plugin and enable the node agent in the `DataProtectionApplication` custom resource (CR). The node agent is a Kubernetes daemonset that hosts data movement modules. These include the Data Mover controller, uploader, and the repository.
23+
24+
.Example `DataProtectionApplication` manifest
25+
[source,yaml]
26+
----
27+
apiVersion: oadp.openshift.io/v1alpha1
28+
kind: DataProtectionApplication
29+
metadata:
30+
name: dpa-sample
31+
spec:
32+
configuration:
33+
nodeAgent:
34+
enable: true <1>
35+
uploaderType: kopia <2>
36+
velero:
37+
defaultPlugins:
38+
- openshift
39+
- aws
40+
- csi <3>
41+
# ...
42+
----
43+
<1> The flag to enable the node agent.
44+
<2> The type of uploader. The possible values are `restic` or `kopia`. The built-in Data Mover uses Kopia as the default uploader mechanism regardless of the value of the `uploaderType` field.
45+
<3> The CSI plugin included in the list of default plugins.
46+
47+
[id="built-in-data-mover-crs"]
48+
== Built-in Data Mover controller and custom resource definitions (CRDs)
49+
50+
The built-in Data Mover feature introduces three new API objects defined as CRDs for managing backup and restore:
51+
52+
* `DataDownload`: Represents a data download of a volume snapshot. The CSI plugin creates one `DataDownload` object per volume to be restored. The `DataDownload` CR includes information about the target volume, the specified Data Mover, the progress of the current data download, the specified backup repository, and the result of the current data download after the process is complete.
53+
54+
* `DataUpload`: Represents a data upload of a volume snapshot. The CSI plugin creates one `DataUpload` object per CSI snapshot. The `DataUpload` CR includes information about the specified snapshot, the specified Data Mover, the specified backup repository, the progress of the current data upload, and the result of the current data upload after the process is complete.
55+
56+
* `BackupRepository`: Represents and manages the lifecycle of the backup repositories. OADP creates a backup repository per namespace when the first CSI snapshot backup or restore for a namespace is requested.
57+
58+

‎backup_and_restore/application_backup_and_restore/installing/oadp-backup-restore-csi-snapshots.adoc

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="oadp-backup-restore-csi-snapshots"]
3+
= Backing up and restoring CSI snapshots
4+
include::_attributes/common-attributes.adoc[]
5+
:context: oadp-backup-restore-csi-snapshots
6+
7+
toc::[]
8+
9+
You can back up and restore persistent volumes by using the OADP 1.3 Data Mover.
10+
11+
include::modules/oadp-1-3-backing-csi-snapshots.adoc[leveloffset=+1]
12+
13+
include::modules/oadp-1-3-restoring-csi-snapshots.adoc[leveloffset=+1]
14+
15+
16+

‎modules/oadp-1-3-backing-csi-snapshots.adoc

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * backup_and_restore/application_backup_and_restore/installing/oadp-backup-restore-csi-snapshots.adoc
4+
:_mod-docs-content-type: PROCEDURE
5+
[id="oadp-1-3-backing-csi-snapshots_{context}"]
6+
= Backing up persistent volumes with CSI snapshots
7+
8+
You can use the OADP Data Mover to back up Container Storage Interface (CSI) volume snapshots to a remote object store.
9+
10+
.Prerequisites
11+
12+
* You have access to the cluster with the `cluster-admin` role.
13+
* You have installed the OADP Operator.
14+
* You have included the CSI plugin and enabled the node agent in the `DataProtectionApplication` custom resource (CR).
15+
* You have an application with persistent volumes running in a separate namespace.
16+
* You have added the `metadata.labels.velero.io/csi-volumesnapshot-class: "true"` key-value pair to the `VolumeSnapshotClass` CR.
17+
18+
.Procedure
19+
20+
. Create a YAML file for the `Backup` object, as in the following example:
21+
+
22+
.Example `Backup` CR
23+
[source,yaml]
24+
----
25+
kind: Backup
26+
apiVersion: velero.io/v1
27+
metadata:
28+
name: backup
29+
namespace: openshift-adp
30+
spec:
31+
csiSnapshotTimeout: 10m0s
32+
defaultVolumesToFsBackup: false
33+
includedNamespaces:
34+
- mysql-persistent
35+
itemOperationTimeout: 4h0m0s
36+
snapshotMoveData: true <1>
37+
storageLocation: default
38+
ttl: 720h0m0s
39+
volumeSnapshotLocations:
40+
- dpa-sample-1
41+
# ...
42+
----
43+
<1> Set to `true` to enable movement of CSI snapshots to remote object storage.
44+
45+
. Apply the manifest:
46+
+
47+
[source,terminal]
48+
----
49+
$ oc create -f backup.yaml
50+
----
51+
+
52+
A `DataUpload` CR is created after the snapshot creation is complete.
53+
54+
.Verification
55+
* Verify that the snapshot data is successfully transferred to the remote object store by monitoring the `status.phase` field of the `DataUpload` CR. Possible values are `In Progress`, `Completed`, `Failed`, or `Canceled`. The object store is configured in the `backupLocations` stanza of the `DataProtectionApplication` CR.
56+
57+
** Run the following command to get a list of all `DataUpload` objects:
58+
+
59+
[source,terminal]
60+
----
61+
$ oc get datauploads -A
62+
----
63+
+
64+
.Example output
65+
[source,terminal]
66+
----
67+
NAMESPACE NAME STATUS STARTED BYTES DONE TOTAL BYTES STORAGE LOCATION AGE NODE
68+
openshift-adp backup-test-1-sw76b Completed 9m47s 108104082 108104082 dpa-sample-1 9m47s ip-10-0-150-57.us-west-2.compute.internal
69+
openshift-adp mongo-block-7dtpf Completed 14m 1073741824 1073741824 dpa-sample-1 14m ip-10-0-150-57.us-west-2.compute.internal
70+
----
71+
72+
** Check the value of the `status.phase` field of the specific `DataUpload` object by running the following command:
73+
+
74+
[source,terminal]
75+
----
76+
$ oc get datauploads <dataupload_name> -o yaml
77+
----
78+
+
79+
.Example output
80+
[source,yaml]
81+
----
82+
apiVersion: velero.io/v2alpha1
83+
kind: DataUpload
84+
metadata:
85+
name: backup-test-1-sw76b
86+
namespace: openshift-adp
87+
spec:
88+
backupStorageLocation: dpa-sample-1
89+
csiSnapshot:
90+
snapshotClass: ""
91+
storageClass: gp3-csi
92+
volumeSnapshot: velero-mysql-fq8sl
93+
operationTimeout: 10m0s
94+
snapshotType: CSI
95+
sourceNamespace: mysql-persistent
96+
sourcePVC: mysql
97+
status:
98+
completionTimestamp: "2023-11-02T16:57:02Z"
99+
node: ip-10-0-150-57.us-west-2.compute.internal
100+
path: /host_pods/15116bac-cc01-4d9b-8ee7-609c3bef6bde/volumes/kubernetes.io~csi/pvc-eead8167-556b-461a-b3ec-441749e291c4/mount
101+
phase: Completed <1>
102+
progress:
103+
bytesDone: 108104082
104+
totalBytes: 108104082
105+
snapshotID: 8da1c5febf25225f4577ada2aeb9f899
106+
startTimestamp: "2023-11-02T16:56:22Z"
107+
----
108+
<1> Indicates that snapshot data is successfully transferred to the remote object store.

‎modules/oadp-1-3-restoring-csi-snapshots.adoc

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * backup_and_restore/application_backup_and_restore/oadp_built_in_data_mover/oadp-backup-restore-csi-snapshots.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="oadp-1-3-restoring-csi-snapshots_{context}"]
7+
= Restoring CSI volume snapshots
8+
9+
You can restore a volume snapshot by creating a `Restore` CR.
10+
11+
[NOTE]
12+
====
13+
You cannot restore Volsync backups from OADP 1.2 with the OAPD 1.3 built-in Data Mover. It is recommended to do a file system backup of all of your workloads with Restic prior to upgrading to OADP 1.3.
14+
====
15+
16+
.Prerequisites
17+
18+
* You have access to the cluster with the `cluster-admin` role.
19+
* You have an OADP `Backup` CR from which to restore the data.
20+
21+
.Procedure
22+
23+
. Create a YAML file for the `Restore` CR, as in the following example:
24+
+
25+
.Example `Restore` CR
26+
[source,yaml]
27+
----
28+
apiVersion: velero.io/v1
29+
kind: Restore
30+
metadata:
31+
name: restore
32+
namespace: openshift-adp
33+
spec:
34+
backupName: <backup>
35+
# ...
36+
----
37+
38+
. Apply the manifest:
39+
+
40+
[source,terminal]
41+
----
42+
$ oc create -f restore.yaml
43+
----
44+
+
45+
A `DataDownload` CR is created when the restore starts.
46+
47+
.Verification
48+
* You can monitor the status of the restore process by checking the `status.phase` field of the `DataDownload` CR. Possible values are `In Progress`, `Completed`, `Failed`, or `Canceled`.
49+
50+
** To get a list of all `DataDownload` objects, run the following command:
51+
+
52+
[source,terminal]
53+
----
54+
$ oc get datadownloads -A
55+
----
56+
+
57+
.Example output
58+
[source,terminal]
59+
----
60+
NAMESPACE NAME STATUS STARTED BYTES DONE TOTAL BYTES STORAGE LOCATION AGE NODE
61+
openshift-adp restore-test-1-sk7lg Completed 7m11s 108104082 108104082 dpa-sample-1 7m11s ip-10-0-150-57.us-west-2.compute.internal
62+
----
63+
64+
** Enter the following command to check the value of the `status.phase` field of the specific `DataDownload` object:
65+
+
66+
[source,terminal]
67+
----
68+
$ oc get datadownloads <datadownload_name> -o yaml
69+
----
70+
+
71+
.Example output
72+
[source,yaml]
73+
----
74+
apiVersion: velero.io/v2alpha1
75+
kind: DataDownload
76+
metadata:
77+
name: restore-test-1-sk7lg
78+
namespace: openshift-adp
79+
spec:
80+
backupStorageLocation: dpa-sample-1
81+
operationTimeout: 10m0s
82+
snapshotID: 8da1c5febf25225f4577ada2aeb9f899
83+
sourceNamespace: mysql-persistent
84+
targetVolume:
85+
namespace: mysql-persistent
86+
pv: ""
87+
pvc: mysql
88+
status:
89+
completionTimestamp: "2023-11-02T17:01:24Z"
90+
node: ip-10-0-150-57.us-west-2.compute.internal
91+
phase: Completed <1>
92+
progress:
93+
bytesDone: 108104082
94+
totalBytes: 108104082
95+
startTimestamp: "2023-11-02T17:00:52Z"
96+
----
97+
<1> Indicates that the CSI snapshot data is successfully restored.

0 commit comments

Comments
 (0)