PBM-1490-Documented the ability to set file size limit for large backups#298
PBM-1490-Documented the ability to set file size limit for large backups#298nastena1606 merged 3 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR documents the ability to set file size limits for large backups in Percona Backup for MongoDB (PBM). The documentation explains how PBM can split large backup files into smaller pieces when they exceed configured size limits.
- Adds documentation for the
maxObjSizeGBconfiguration parameter across all supported storage types - Creates a new feature documentation page explaining the split-merge backup functionality
- Updates navigation to include the new documentation page
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| mkdocs-base.yml | Adds navigation entry for the new split-merge backup documentation |
| docs/reference/configuration-options.md | Documents the maxObjSizeGB parameter for S3, GCS, filesystem, and Azure storage types |
| docs/features/split-merge-backup.md | New comprehensive documentation explaining how large backup file management works |
| docs/details/storage-configuration.md | Minor grammar fix in existing content |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| numMaxRetries: 3 | ||
| minRetryDelay: 30ms | ||
| maxRetryDelay: 5m | ||
| maxObjSizeGB: 5018 |
There was a problem hiding this comment.
The maxObjSizeGB parameter appears under the filesystem storage section but should be at the same indentation level as filesystem to be a sibling of the storage type configuration, not nested within it.
modified: docs/details/storage-configuration.md
new file: docs/features/split-merge-backup.md
modified: mkdocs-base.yml
modified: docs/reference/configuration-options.md
39ae623 to
4b0824a
Compare
boris-ilijic
left a comment
There was a problem hiding this comment.
We need to change type int->float64. The other comments are optional.
docs/features/split-merge-backup.md
Outdated
|
|
||
| !!! admonition "Version added: [2.11.0](../release-notes/2.11.0.md) | ||
|
|
||
| As your database grows, so do your backups. Eventually, a backup file may become so large that it exceeds the maximum object size limit of your cloud or local storage. When this happens, Percona Backup for MongoDB (PBM) can't upload the file, which can disrupt your backup strategy. |
There was a problem hiding this comment.
Eventually, a backup file (typically collection or index) ....
|
|
||
| ### storage.s3.maxObjSizeGB | ||
|
|
||
| *Type*: int <br> |
|
|
||
| ### storage.gcs.maxObjSizeGB | ||
|
|
||
| *Type*: int <br> |
|
|
||
| ### storage.filesystem.maxObjSizeGB | ||
|
|
||
| *Type*: int <br> |
|
|
||
| ### storage.azure.maxObjSizeGB | ||
|
|
||
| *Type*: int <br> |
docs/features/split-merge-backup.md
Outdated
| | **Azure Blob Storage** | 190 TB | | ||
| | **Filesystem storage** | 4.9 TB | | ||
|
|
||
| To ensure that PBM successfully uploads backups, you can configure the maximum size for backup files for the storage you use. Define the file size in GB for the `maxObjSizeGB` configuration parameter. |
There was a problem hiding this comment.
My recommendation is to add warning something like this:
We don't recommend modifying the maxObjSizeGB param. The default value is sufficient for most use cases, and setting it to a lower value can be dangerous as it may cause a single backup file to be split into a large number of smaller files.
There was a problem hiding this comment.
But what is the implication, does it affect performance or PBM won't be able to merge all those chunks together?
There was a problem hiding this comment.
Yes, mostly performance degradation. Also on the storage it'll be large number of files instead of single file of few of them, so it'll be harder to manage those (e.g. copy or browse), but I wouldn't mention that details.
2848435 to
b55ee7f
Compare
modified: docs/details/storage-configuration.md
new file: docs/features/split-merge-backup.md
modified: mkdocs-base.yml
modified: docs/reference/configuration-options.md