Merge pull request #69704 from RichardHoch/OADP_130_perform_scale

OADP-3261: OADP 1.3 Performance & Scale

Author: stevsmit

_attributes/common-attributes.adoc

@@ -41,8 +41,10 @@ endif::openshift-origin[]
 :insights-advisor-url: link:https://console.redhat.com/openshift/insights/advisor/[Insights Advisor]
 :hybrid-console: Red Hat Hybrid Cloud Console
 :hybrid-console-second: Hybrid Cloud Console
+// OADP attributes
 :oadp-first: OpenShift API for Data Protection (OADP)
 :oadp-full: OpenShift API for Data Protection
+:oadp-short: OADP
 :oc-first: pass:quotes[OpenShift CLI (`oc`)]
 :product-registry: OpenShift image registry
 :rh-storage-first: Red Hat OpenShift Data Foundation
@@ -253,3 +255,7 @@ endif::[]
 :coo-first: Cluster Observability Operator (COO)
 :coo-full: Cluster Observability Operator
 :coo-short: COO
+//ODF
+:odf-first: Red Hat OpenShift Data Foundation (ODF)
+:odf-full: Red Hat OpenShift Data Foundation
+:odf-short: ODF

backup_and_restore/application_backup_and_restore/installing/about-installing-oadp.adoc

@@ -57,3 +57,4 @@ include::modules/about-installing-oadp-on-multiple-namespaces.adoc[leveloffset=+
 * xref:../../../operators/understanding/olm/olm-understanding-olm.adoc#olm-csv_olm-understanding-olm[Cluster service version]
 
 include::modules/oadp-velero-cpu-memory-requirements.adoc[leveloffset=+1]
+include::modules/oadp-backup-restore-for-large-usage.adoc[leveloffset=+2]

backup_and_restore/application_backup_and_restore/installing/installing-oadp-ocs.adoc

@@ -37,6 +37,7 @@ include::modules/oadp-secrets-for-different-credentials.adoc[leveloffset=+2]
 You can configure the Data Protection Application by setting Velero resource allocations or enabling self-signed CA certificates.
 
 include::modules/oadp-setting-resource-limits-and-requests.adoc[leveloffset=+2]
+include::modules/oadp-odf-cpu-memory-requirements.adoc[leveloffset=+3]
 include::modules/oadp-self-signed-certificate.adoc[leveloffset=+2]
 
 include::modules/oadp-installing-dpa-1-2-and-earlier.adoc[leveloffset=+1]

modules/oadp-backup-restore-for-large-usage.adoc

@@ -0,0 +1,31 @@
+// Module included in the following assemblies:
+// * backup_and_restore/application_backup_and_restore/installing/about-installing-oadp.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="oadp-backup-restore-for-large-usage_{context}"]
+= NodeAgent CPU for large usage
+
+Testing shows that increasing `NodeAgent` CPU can significantly improve backup and restore times when using {oadp-first}.
+
+[IMPORTANT]
+====
+It is not recommended to use Kopia without limits in production environments on nodes running production workloads due to Kopia’s aggressive consumption of resources. However, running Kopia with limits that are too low results in CPU limiting and slow backups and restore situations. Testing showed that running Kopia with 20 cores and 32 Gi memory supported backup and restore operations of over 100 GB of data, multiple namespaces, or over 2000 pods in a single namespace.
+====
+
+Testing detected no CPU limiting or memory saturation with these resource specifications.
+
+You can set these limits in Ceph MDS pods by following the procedure in https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.14/html/troubleshooting_openshift_data_foundation/changing-resources-for-the-openshift-data-foundation-components_rhodf#changing_the_cpu_and_memory_resources_on_the_rook_ceph_pods[Changing the CPU and memory resources on the rook-ceph pods].
+
+You need to add the following lines to the storage cluster Custom Resource (CR) to set the limits:
+
+[source,yaml]
+----
+   resources:
+     mds:
+       limits:
+         cpu: "3"
+         memory: 128Gi
+       requests:
+         cpu: "3"
+         memory: 8Gi
+----

modules/oadp-odf-cpu-memory-requirements.adoc

@@ -0,0 +1,26 @@
+// Module included in the following assemblies:
+//
+// * backup_and_restore/application_backup_and_restore/installing/installing-oadp-ocs.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="oadp-odf-cpu-memory-requirements_{context}"]
+= Adjusting Ceph CPU and memory requirements based on collected data
+
+The following recommendations are based on observations of performance made in the scale and performance lab. The changes are specifically related to {odf-first}. If working with {odf-short}, consult the appropriate tuning guides for official recommendations.
+
+[id="oadp-odf-config-cpu-memory-requirements_{context}"]
+== CPU and memory requirement for configurations
+
+Backup and restore operations require large amounts of CephFS `PersistentVolumes` (PVs). To avoid Ceph MDS pods restarting with an `out-of-memory` (OOM) error, the following configuration is suggested:
+
+|===
+| Configuration types | Request | Max limit
+
+| CPU
+| Request changed to 3
+| Max limit to 3
+
+| Memory
+| Request changed to 8 Gi
+| Max limit to 128 Gi
+|===