Merge branch 'master' into xn4p4lm-host-check-formatting-update

Author: xN4P4LM

backup.config-example

@@ -16,10 +16,18 @@ GHE_DATA_DIR="data"
 # be available for the past N days ...
 GHE_NUM_SNAPSHOTS=10
 
-# Pruning snapshots can be scheduled outside of the backup process. If set to 'yes'
-# ghe-pruning-snapshots will need to be invoked separately via cron
+# Pruning snapshots can be scheduled outside of the backup process.
+# If set to 'yes', snapshots will not be pruned by ghe-backup.
+# Instead, ghe-pruning-snapshots will need to be invoked separately via cron
 #GHE_PRUNING_SCHEDULED=yes
 
+# If --incremental is used to generate incremental MySQL backups with ghe-backup,
+# then you need to specify how many cycles of full and incremental backups will be 
+# performed before the next full backup is created.
+# For example, if `GHE_INCREMENTAL_BACKUP_MAX` is set to 14, backup-utils will 
+# run 1 full backup and then 13 incremental backups before performing another full backup on the next cycle.
+#GHE_INCREMENTAL_BACKUP_MAX=14
+
 # If GHE_SKIP_CHECKS is set to true (or if --skip-checks is used with ghe-backup) then ghe-host-check
 # disk space validation and software version checks on the backup-host will be disabled.
 #GHE_SKIP_CHECKS=false

docs/incremental-mysql-backups-and-restores.md

@@ -0,0 +1,38 @@
+# Incremental MySQL Backups and Restores
+
+Customers who have large MySQL databases who wish to save storage space can use the `--incremental` flag with `ghe-backup` and `ghe-restore`.
+Using this flag performs backups for other parts of GHES as normal, but only performs a MySQL backup of the changes to the database from the previous snapshot. 
+For larger databases this can conserve a lot of storage space for backups.
+
+## Configuring number of backups
+
+In your backup.config file you will need to set the variable `GHE_INCREMENTAL_BACKUP_MAX`.
+This variable determines how many cycles of full and incremental backups will be performed before the next full backup is created.
+For example, if `GHE_INCREMENTAL_BACKUP_MAX` is set to 14, backup-utils will run 1 full backup and then 13 incremental backups before performing another full backup on the next cycle.
+
+Incremental backups require the previous snapshot backups before them to work.
+This means they do not follow the pruning strategy based on `GHE_NUM_SNAPSHOTS`.
+
+## Performing incremental backups
+
+To perform incremental backups:
+
+`bin/ghe-backup --incremental`
+
+the program will detect whether it needs to performa full or incremental snapshot based on what is currently in `GHE_DATA_DIR`. 
+
+To see what snapshots are part of your full and incremental backups, you can reference `GHE_DATA_DIR/inc_full_backup` and `GHE_DATA_DIR/inc_snapshot_data`, respectively.
+
+## Performing incremental restores
+
+To perform incremental restores:
+
+`bin/ghe-restore --incremental -s <snapshot-id>`
+
+The program will use the MySQL folders from each previous incremental backup and the full backup to restore the database.
+
+:warning: Incremental restores require the other snapshots in the cycle to complete a restore. Erasing snapshot directories that are part of a cycle corrupts the restore and makes it impossible to restore for the MySQL database.
+
+### Previous cycles
+
+To ensure there is a rolling window of mySQL backups, incremental MySQL backups from the cycle before the current one are kept.  Those snapshots are pre-pended with `inc_previous`. To perform a restore from there, just use the full directory name for the snapshot id.

docs/scheduling-backups.md

@@ -4,7 +4,8 @@ Regular backups should be scheduled using `cron(8)` or similar command
 scheduling service on the backup host. The backup frequency will dictate the
 worst case [recovery point objective (RPO)][1] in your backup plan. We recommend
 hourly backups at the least.
-## Example scheduling usage
+
+## Example scheduling of backups
 
 The following examples assume the Backup Utilities are installed under
 `/opt/backup-utils`. The crontab entry should be made under the same user that
@@ -32,13 +33,14 @@ [email protected]
 0 0 * * * /opt/backup-utils/bin/ghe-backup -v 1>>/opt/backup-utils/backup.log 2>&1
 ```
 
-### Example snapshot pruning 
+## Example snapshot pruning 
 
 By default all expired and incomplete snapshots are deleted at the end of the main
 backup process `ghe-backup`. If pruning these snapshots takes a long time you can
 choose to disable the pruning process from the backup run and schedule it separately.
 This can be achieved by enabling the `GHE_PRUNING_SCHEDULED` option in `backup.config`.
-Please note that this option is only avilable for `backup-utils` >= v3.10.0, if this option is enabled you will need to schedule the pruning script `ghe-prune-snapshots` using `cron` or a similar command scheduling service on the backup host.
+Please note that this option is only avilable for `backup-utils` >= `v3.10.0`.
+If this option is enabled you will need to schedule the pruning script `ghe-prune-snapshots` using `cron` or a similar command scheduling service on the backup host.
 
 To schedule daily snapshot pruning, use:
 

docs/usage.md

@@ -111,5 +111,9 @@ GitHub Actions enabled, the following steps are required:
 
 Please refer to [GHES Documentation](https://docs.github.com/en/enterprise-server/admin/github-actions/advanced-configuration-and-troubleshooting/backing-up-and-restoring-github-enterprise-server-with-github-actions-enabled) for more details.
 
+## Incremental MySQL Backups and Restores
+
+If you are interested in performing incremental backups of the MySQL data in your GitHub Enterprise Server instance, see [Incremental MySQL Backups and Restores](incremental-mysql-backups-and-restores.md) for details.
+
 [1]: https://github.com/github/backup-utils/blob/master/docs/getting-started.md
 [2]: requirements.md