PBM-1490-Documented the ability to set file size limit for large backups

modified:   docs/details/storage-configuration.md
        new file:   docs/features/split-merge-backup.md
        modified:   mkdocs-base.yml
        modified: docs/reference/configuration-options.md

Author: nastena1606

docs/details/storage-configuration.md

@@ -18,7 +18,7 @@ Percona Backup for MongoDB (PBM) saves backup data to a designated directory on
 
 The oplog entries ensure backup consistency, and the end time of the oplog slice(s) is the data-consistent point in time of a backup snapshot.
 
-Using the [`pbm list`](../reference/pbm-commands.md#pbm-list) or [`pbm status`](../reference/pbm-commands.md#pbm-status) commands, you can scan the backup directory to find existing backups, even if never used PBM on your computer before.
+Using the [`pbm list`](../reference/pbm-commands.md#pbm-list) or [`pbm status`](../reference/pbm-commands.md#pbm-status) commands, you can scan the backup directory to find existing backups, even if you never used PBM on your computer before.
 
 ## Supported storage types
 

docs/features/split-merge-backup.md

@@ -0,0 +1,43 @@
+# Manage large backup files upload 
+
+!!! admonition "Version added: [2.11.0](../release-notes/2.11.0.md)
+
+As your database grows, so do your backups. Eventually, a backup file may become so large that it exceeds the maximum object size limit of your cloud or local storage. When this happens, Percona Backup for MongoDB (PBM) can't upload the file, which can disrupt your backup strategy.
+
+The following table provides default size limits for the supported backup storages:
+
+| Storage | Default size limit|
+| :--- | :--- |
+| **AWS S3** | 4.9 TB |
+| **GCS** | 4.9 TB |
+| **Azure Blob Storage** | 190 TB |
+| **Filesystem storage** | 4.9 TB |
+
+To ensure that PBM successfully uploads backups, you can configure the maximum size for backup files for the storage you use. Define the file size in GB for the `maxObjSizeGB` configuration parameter.
+
+This is the configuration example for the AWS S3 storage:
+
+```yaml
+storage:
+  type: s3
+  s3:
+    region: us-west-2
+    bucket: pbm-test-bucket
+    prefix: data/pbm/backup
+    credentials:
+      access-key-id: <your-access-key-id-here>
+      secret-access-key: <your-secret-key-here>
+  maxObjSizeGB: 100
+```
+
+## How it works
+
+When a backup file exceeds the configured size limit, PBM does the following:
+
+* Splits the file into pieces, each of which doesn't exceed the defined size limit. PBM respects your compression settings, so compressed backups stay compressed after the split
+* Names each piece by adding the `pbmpart{number}` token to the filename. This lets PBM identify and manage all the pieces of a single backup.
+* Uploads the pieces to the storage
+
+When reading data for an operation like a restore, PBM automatically reverses this process: it retrieves all the pieces from storage, merges them back into a single file, and proceeds with the command. This makes the entire process transparent to your application.
+
+With this ability to set the maximum file size, you can future-proof your backup strategy and gain peace of mind knowing your data is always safe and recoverable. 

docs/reference/configuration-options.md

@@ -38,6 +38,7 @@ storage:
       numMaxRetries: 3
       minRetryDelay: 30ms
       maxRetryDelay: 5m
+    maxObjSizeGB: 5018
 ```
 
 ### storage.s3.provider
@@ -246,6 +247,14 @@ The minimum time to wait before the next retry, specified as a *time.Duration*.
 
 The maximum time to wait before the next retry, specified as a *time.Duration*. Units like ms, s, etc., are supported. Defaults to nanoseconds if no unit is provided. 
 
+### storage.s3.maxObjSizeGB
+
+*Type*: int <br>
+*Required*: NO <br>
+*Default*: 5018
+
+The maximum file size to be stored on the backup storage. If the file to upload exceeds this limit, PBM splits it in pieces, each of which falls within the limit. Read more about [Managing large backup files](../features/split-merge-backup.md).
+
 ## GCS type storage options
 
 ```yaml
@@ -260,6 +269,7 @@ storage:
       privateKey: <your-private-key-here>
       hmacAccessKey: <your-HMAC-key-here>
       hmacSecret: <your-HMAC-secret-here>
+    maxObjSizeGB: 5018
 ```
 
 ### storage.gcs.bucket
@@ -332,13 +342,22 @@ The maximum amount of time between retries, in seconds. Defaults to 30 sec.
 
 Each time PBM fails and tries again, it increases the wait time by multiplying it by this number (usually 2). For example, if the first wait time is 1 second, the next will be 2 seconds, then 4 seconds, and so on, until it reaches the maximum. Default value is 2 sec.
 
+### storage.gcs.maxObjSizeGB
+
+*Type*: int <br>
+*Required*: NO <br>
+*Default*: 5018
+
+The maximum file size to be stored on the backup storage. If the file to upload exceeds this limit, PBM splits it in pieces, each of which falls within the defined limit. Read more about [Managing large backup files](../features/split-merge-backup.md).
+
 ## Filesystem storage options
 
 ```yaml
 storage:
   type: filesystem
   filesystem:
     path: <string>
+  maxObjSizeGB: 5018
 ```
 
 ### storage.filesystem.path
@@ -348,6 +367,14 @@ storage:
 
 The path to the backup directory
 
+### storage.filesystem.maxObjSizeGB
+
+*Type*: int <br>
+*Required*: NO <br>
+*Default*: 5018
+
+The maximum file size to be stored on the backup storage. If the file to upload exceeds this limit, PBM splits it in pieces, each of which falls within the defined limit. Read more about [Managing large backup files](../features/split-merge-backup.md).
+
 ## Microsoft Azure Blob storage options
 
 ```yaml
@@ -360,6 +387,7 @@ storage:
     prefix: <string>
     credentials:
       key: <your-access-key>
+    maxObjSizeGB: 194560
 ```
 
 ### storage.azure.account
@@ -397,3 +425,10 @@ The path to the data directory in the bucket. If undefined, backups are stored i
 
 Your access key to authorize access to data in your storage account.
 
+### storage.azure.maxObjSizeGB
+
+*Type*: int <br>
+*Required*: NO <br>
+*Default*: 194560
+
+The maximum file size to be stored on the backup storage. If the file to upload exceeds this limit, PBM splits it in pieces, each of which falls within the defined limit. Read more about [Managing large backup files](../features/split-merge-backup.md).

mkdocs-base.yml

@@ -233,6 +233,7 @@ nav:
       - details/filesystem-storage.md
       - details/azure.md
       - features/multi-storage.md
+      - features/split-merge-backup.md
   - PBM commands: reference/pbm-commands.md
   - Configuration file: 
       - Percona Backup for MongoDB configuration in a cluster: reference/config.md