Skip to content

Commit 9ea34f5

Browse files
committed
Add MongoDB Atlas Support
1 parent 1d53785 commit 9ea34f5

File tree

2 files changed

+115
-31
lines changed

2 files changed

+115
-31
lines changed

README.md

Lines changed: 49 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ Currently backs up CouchDB, InfluxDB, MySQL, MongoDB, Postgres, Redis servers.
5959
- [Manual Backups](#manual-backups)
6060
- [Restoring Databases](#restoring-databases)
6161
- [Custom Scripts](#custom-scripts)
62+
- [Pre Backup](#pre-backup)
63+
- [Post backup](#post-backup)
6264
- [Support](#support)
6365
- [Usage](#usage)
6466
- [Bugfixes](#bugfixes)
@@ -104,10 +106,11 @@ Images are built primarily for `amd64` architecture, and may also include builds
104106
### Persistent Storage
105107

106108
The following directories are used for configuration and can be mapped for persistent storage.
107-
| Directory | Description |
108-
| ------------------------ | ---------------------------------------------------------------------------------- |
109-
| `/backup` | Backups |
110-
| `/assets/custom-scripts` | *Optional* Put custom scripts in this directory to execute after backup operations |
109+
| Directory | Description |
110+
| ---------------------------- | ----------------------------------------------------------------------------------- |
111+
| `/backup` | Backups |
112+
| `/assets/custom-scripts/pre` | *Optional* Put custom scripts in this directory to execute before backup operations |
113+
| `/assets/custom-scripts` | *Optional* Put custom scripts in this directory to execute after backup operations |
111114

112115
### Environment Variables
113116

@@ -130,21 +133,24 @@ Be sure to view the following repositories to understand all the customizable op
130133
| `MANUAL_RUN_FOREVER` | `TRUE` or `FALSE` if you wish to try to make the container exit after the backup | `TRUE` |
131134
| `TEMP_LOCATION` | Perform Backups and Compression in this temporary directory | `/tmp/backups/` |
132135
| `DEBUG_MODE` | If set to `true`, print copious shell script messages to the container log. Otherwise only basic messages are printed. | `FALSE` |
136+
| `PRE_SCRIPT` | Fill this variable in with a command to execute post the script backing up | |
133137
| `POST_SCRIPT` | Fill this variable in with a command to execute post the script backing up | |
134-
| `SPLIT_DB` | For each backup, create a new archive. `TRUE` or `FALSE` (MySQL and Postgresql Only) | `TRUE`
138+
| `SPLIT_DB` | For each backup, create a new archive. `TRUE` or `FALSE` (MySQL and Postgresql Only) | `TRUE` |
135139

136140
### Database Specific Options
137-
| Parameter | Description | Default |
138-
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
139-
| `DB_AUTH` | (Mongo Only - Optional) Authentication Database | |
140-
| `DB_TYPE` | Type of DB Server to backup `couch` `influx` `mysql` `pgsql` `mongo` `redis` `sqlite3` | |
141-
| `DB_HOST` | Server Hostname e.g. `mariadb`. For `sqlite3`, full path to DB file e.g. `/backup/db.sqlite3` | |
142-
| `DB_NAME` | Schema Name e.g. `database` or `ALL` to backup all databases the user has access to. Backup multiple by seperating with commas eg `db1,db2` | |
143-
| `DB_NAME_EXCLUDE` | If using `ALL` - use this as to exclude databases seperated via commas from being backed up | |
144-
| `DB_USER` | username for the database(s) - Can use `root` for MySQL | |
145-
| `DB_PASS` | (optional if DB doesn't require it) password for the database | |
146-
| `DB_PORT` | (optional) Set port to connect to DB_HOST. Defaults are provided | varies |
147-
| `INFLUX_VERSION` | What Version of Influx are you backing up from `1`.x or `2` series - AMD64 and ARM64 only for `2` | |
141+
| Parameter | Description | Default |
142+
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | --------- |
143+
| `DB_AUTH` | (Mongo Only - Optional) Authentication Database | |
144+
| `DB_TYPE` | Type of DB Server to backup `couch` `influx` `mysql` `pgsql` `mongo` `redis` `sqlite3` | |
145+
| `DB_HOST` | Server Hostname e.g. `mariadb`. For `sqlite3`, full path to DB file e.g. `/backup/db.sqlite3` | |
146+
| `DB_NAME` | Schema Name e.g. `database` or `ALL` to backup all databases the user has access to. Backup multiple by seperating with commas eg `db1,db2` | |
147+
| `DB_NAME_EXCLUDE` | If using `ALL` - use this as to exclude databases seperated via commas from being backed up | |
148+
| `DB_USER` | username for the database(s) - Can use `root` for MySQL | |
149+
| `DB_PASS` | (optional if DB doesn't require it) password for the database | |
150+
| `DB_PORT` | (optional) Set port to connect to DB_HOST. Defaults are provided | varies |
151+
| `INFLUX_VERSION` | What Version of Influx are you backing up from `1`.x or `2` series - AMD64 and ARM64 only for `2` | |
152+
| `MONGO_HOST_TYPE` | Connect to regular `mongodb` or `atlas` | `mongodb` |
153+
| | You can also skip this and override the uri prefix with `MONGO_URI_PREFIX=mongodb+srv://` or whatever you would like | |
148154

149155
#### For Influx DB2:
150156
Your Organization will be mapped to `DB_USER` and your root token will need to be mapped to `DB_PASS`. You may use `DB_NAME=ALL` to backup the entire set of databases. For `DB_HOST` use syntax of `http(s)://db-name`
@@ -231,7 +237,33 @@ If you only enter some of the arguments you will be prompted to fill them in.
231237

232238
### Custom Scripts
233239

234-
If you want to execute a custom script at the end of backup, you can drop bash scripts with the extension of `.sh` in this directory. See the following example to utilize:
240+
#### Pre Backup
241+
If you want to execute a custom script before a backup starts, you can drop bash scripts with the extension of `.sh` in `/assets/custom/pre`. See the following example to utilize:
242+
243+
````bash
244+
$ cat pre-script.sh
245+
##!/bin/bash
246+
247+
# #### Example Pre Script
248+
# #### $2=DB_TYPE (Type of Backup)
249+
# #### $3=DB_HOST (Backup Host)
250+
# #### #4=DB_NAME (Name of Database backed up
251+
# #### $5=BACKUP START TIME (Seconds since Epoch)
252+
# #### $8=BACKUP FILENAME (Filename)
253+
254+
echo "${2} Backup Starting on ${3} for ${4} on ${5} ending ${6} for a duration of ${7} seconds. Filename: ${8} Size: ${9} bytes MD5: ${10}"
255+
````
256+
257+
## script EXIT_CODE DB_TYPE DB_HOST DB_NAME STARTEPOCH BACKUP_FILENAME
258+
${f} "${exit_code}" "${dbtype}" "${dbhost}" "${dbname}" "${backup_start_time}" "${target}"
259+
260+
261+
Outputs the following on the console:
262+
263+
`mysql Backup Starting on example-db for example on 1647370800. Filename: mysql_example_example-db_202200315-000000.sql.bz2
264+
265+
#### Post backup
266+
If you want to execute a custom script at the end of backup, you can drop bash scripts with the extension of `.sh` in `/assets/custom`. See the following example to utilize:
235267

236268
````bash
237269
$ cat post-script.sh

install/assets/functions/10-db-backup

Lines changed: 66 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,10 @@ bootstrap_variables() {
1010
;;
1111
influx* )
1212
dbtype=influx
13-
DB_PORT=${DB_PORT:-8088}
13+
case "${INFLUX_VERSION}" in
14+
1) DB_PORT=${DB_PORT:-8088} ;;
15+
2) DB_PORT=${DB_PORT:-8086} ;;
16+
esac
1417
file_env 'DB_USER'
1518
file_env 'DB_PASS'
1619
sanity_var INFLUX_VERSION "What InfluxDB version you are backing up from '1' or '2'"
@@ -78,9 +81,10 @@ bootstrap_variables() {
7881
}
7982

8083
backup_couch() {
81-
pre_dbbackup
84+
prepare_dbbackup
8285
target=couch_${DB_NAME}_${DB_HOST#*//}_${now}.txt
8386
compression
87+
pre_dbbackup ${DB_NAME}
8488
print_notice "Dumping CouchDB database: '${DB_NAME}' ${compression_string}"
8589
curl -sSL -X GET ${DB_HOST}:${DB_PORT}/${DB_NAME}/_all_docs?include_docs=true ${compress_cmd} | $compress_cmd > "${TEMP_LOCATION}"/"${target}"
8690
exit_code=$?
@@ -101,10 +105,11 @@ backup_influx() {
101105
case "${INFLUX_VERSION,,}" in
102106
1 )
103107
for db in ${db_names}; do
104-
pre_dbbackup
108+
prepare_dbbackup
105109
if [ "${db}" != "justbackupeverything" ] ; then bucket="-db ${db}" ; else db=all ; fi
106110
target=influx_${db}_${DB_HOST#*//}_${now}
107111
compression
112+
pre_dbbackup $db
108113
print_notice "Dumping Influx database: '${db}'"
109114
influxd backup ${influx_compression} ${bucket} -portable -host ${DB_HOST}:${DB_PORT} ${EXTRA_OPTS} "${TEMP_LOCATION}"/"${target_dir}"
110115
exit_code=$?
@@ -119,10 +124,11 @@ backup_influx() {
119124
;;
120125
2 )
121126
for db in ${db_names}; do
122-
pre_dbbackup
127+
prepare_dbbackup
123128
if [ "${db}" != "justbackupeverything" ] ; then bucket="--bucket $db" ; else db=all ; fi
124129
target=influx2_${db}_${DB_HOST#*//}_${now}
125130
compression
131+
pre_dbbackup $db
126132
print_notice "Dumping Influx2 database: '${db}'"
127133
influx backup --org ${DB_USER} ${bucket} --host ${DB_HOST}:${DB_PORT} --token ${DB_PASS} ${EXTRA_OPTS} --compression none "${TEMP_LOCATION}"/"${target_dir}"
128134
exit_code=$?
@@ -138,16 +144,22 @@ backup_influx() {
138144
}
139145

140146
backup_mongo() {
141-
pre_dbbackup
147+
prepare_dbbackup
142148
if [ "${ENABLE_COMPRESSION,,}" = "none" ] || [ "${ENABLE_COMPRESSION,,}" = "false" ] ; then
143149
target=${dbtype}_${DB_NAME,,}_${DB_HOST,,}_${now}.archive
144150
else
145151
target=${dbtype}_${DB_NAME,,}_${DB_HOST,,}_${now}.archive.gz
146152
mongo_compression="--gzip"
147153
compression_string="and compressing with gzip"
148154
fi
155+
if [ "${MONGO_HOST_TYPE,,}" = "atlas" ] ; then
156+
MONGO_URI_PREFIX=${MONGO_URI_PREFIX:-"mongodb+srv://"}
157+
else
158+
MONGO_URI_PREFIX=${MONGO_URI_PREFIX:-"mongodb://"}
159+
fi
160+
pre_dbbackup "${DB_NAME}"
149161
print_notice "Dumping MongoDB database: '${DB_NAME}' ${compression_string}"
150-
mongodump --archive=${TEMP_LOCATION}/${target} ${mongo_compression} --host ${DB_HOST} --port ${DB_PORT} ${MONGO_USER_STR}${MONGO_PASS_STR}${MONGO_AUTH_STR}${MONGO_DB_STR} ${EXTRA_OPTS}
162+
mongodump --archive=${TEMP_LOCATION}/${target} ${mongo_compression} --uri="${MONGO_URI_PREFIX}${DB_HOST}:${DB_PORT}" ${MONGO_USER_STR}${MONGO_PASS_STR}${MONGO_AUTH_STR}${MONGO_DB_STR} ${EXTRA_OPTS}
151163
exit_code=$?
152164
check_exit_code $target
153165
generate_checksum
@@ -156,9 +168,10 @@ backup_mongo() {
156168
}
157169

158170
backup_mssql() {
159-
pre_dbbackup
171+
prepare_dbbackup
160172
target=mssql_${DB_NAME,,}_${DB_HOST,,}_${now}.bak
161173
compression
174+
pre_dbbackup "${DB_NAME}"
162175
print_notice "Dumping MSSQL database: '${DB_NAME}'"
163176
/opt/mssql-tools/bin/sqlcmd -E -C -S ${DB_HOST}\,${DB_PORT} -U ${DB_USER} -P ${DB_PASS} –Q "BACKUP DATABASE \[${DB_NAME}\] TO DISK = N'${TEMP_LOCATION}/${target}' WITH NOFORMAT, NOINIT, NAME = '${DB_NAME}-full', SKIP, NOREWIND, NOUNLOAD, STATS = 10"
164177
exit_code=$?
@@ -194,9 +207,10 @@ backup_mysql() {
194207

195208
if var_true "${SPLIT_DB}" ; then
196209
for db in ${db_names} ; do
197-
pre_dbbackup
210+
prepare_dbbackup
198211
target=mysql_${db}_${DB_HOST,,}_${now}.sql
199212
compression
213+
pre_dbbackup $db
200214
print_notice "Dumping MySQL/MariaDB database: '${db}' ${compression_string}"
201215
mysqldump --max-allowed-packet=${MYSQL_MAX_ALLOWED_PACKET} -h ${DB_HOST} -P ${DB_PORT} -u${DB_USER} ${single_transaction} ${stored_procedures} ${EXTRA_OPTS} --databases $db | $compress_cmd > "${TEMP_LOCATION}"/"${target}"
202216
exit_code=$?
@@ -207,9 +221,10 @@ backup_mysql() {
207221
done
208222
else
209223
print_debug "Not splitting database dumps into their own files"
210-
pre_dbbackup
224+
prepare_dbbackup
211225
target=mysql_all_${DB_HOST,,}_${now}.sql
212226
compression
227+
pre_dbbackup all
213228
print_notice "Dumping all MySQL / MariaDB databases: '$(echo ${db_names} | xargs | tr ' ' ',')' ${compression_string}"
214229
mysqldump --max-allowed-packet=${MYSQL_MAX_ALLOWED_PACKET} -h ${DB_HOST} -P ${DB_PORT} -u${DB_USER} ${single_transaction} ${stored_procedures} ${EXTRA_OPTS} --databases $(echo ${db_names} | xargs) | $compress_cmd > "${TEMP_LOCATION}"/"${target}"
215230
exit_code=$?
@@ -241,9 +256,10 @@ backup_pgsql() {
241256

242257
if var_true "${SPLIT_DB}" ; then
243258
for db in ${db_names} ; do
244-
pre_dbbackup
259+
prepare_dbbackup
245260
target=pgsql_${db}_${DB_HOST,,}_${now}.sql
246261
compression
262+
pre_dbbackup $db
247263
print_notice "Dumping PostgresSQL database: '${db}' ${compression_string}"
248264
pg_dump -h ${DB_HOST} -p ${DB_PORT} -U ${DB_USER} $db ${EXTRA_OPTS} | $compress_cmd > ${TEMP_LOCATION}/${target}
249265
exit_code=$?
@@ -254,9 +270,10 @@ backup_pgsql() {
254270
done
255271
else
256272
print_debug "Not splitting database dumps into their own files"
257-
pre_dbbackup
273+
prepare_dbbackup
258274
target=pgsql_all_${DB_HOST,,}_${now}.sql
259275
compression
276+
pre_dbbackup all
260277
print_notice "Dumping all PostgreSQL databases: '$(echo ${db_names} | xargs | tr ' ' ',')' ${compression_string}"
261278
tmp_db_names=$(psql -h ${DB_HOST} -U ${DB_USER} -p ${DB_PORT} -d ${authdb} -c 'COPY (SELECT datname FROM pg_database WHERE datistemplate = false) TO STDOUT;' )
262279
for r_db_name in $(echo $db_names | xargs); do
@@ -276,7 +293,7 @@ backup_pgsql() {
276293
}
277294

278295
backup_redis() {
279-
pre_dbbackup
296+
prepare_dbbackup
280297
print_notice "Dumping Redis - Flushing Redis Cache First"
281298
target=redis_all_${DB_HOST,,}_${now}.rdb
282299
echo bgsave | silent redis-cli -h ${DB_HOST} -p ${DB_PORT} ${REDIS_PASS_STR} --rdb ${TEMP_LOCATION}/${target} ${EXTRA_OPTS}
@@ -296,6 +313,7 @@ backup_redis() {
296313
done
297314
target_original=${target}
298315
compression
316+
pre_dbbackup all
299317
$compress_cmd "${TEMP_LOCATION}/${target_original}"
300318
check_exit_code $target
301319
generate_checksum
@@ -304,11 +322,12 @@ backup_redis() {
304322
}
305323

306324
backup_sqlite3() {
307-
pre_dbbackup
325+
prepare_dbbackup
308326
db=$(basename "${DB_HOST}")
309327
db="${db%.*}"
310328
target=sqlite3_${db}_${now}.sqlite3
311329
compression
330+
pre_dbbackup $db
312331
print_notice "Dumping sqlite3 database: '${DB_HOST}' ${compression_string}"
313332
sqlite3 "${DB_HOST}" ".backup '${TEMP_LOCATION}/backup.sqlite3'"
314333
exit_code=$?
@@ -623,14 +642,47 @@ move_dbbackup() {
623642
rm -rf "${TEMP_LOCATION}"/*
624643
}
625644

626-
pre_dbbackup() {
645+
prepare_dbbackup() {
627646
dbbackup_start_time=$(date +"%s")
628647
now=$(date +"%Y%m%d-%H%M%S")
629648
now_time=$(date +"%H:%M:%S")
630649
now_date=$(date +"%Y-%m-%d")
631650
target=${dbtype}_${DB_NAME,,}_${DB_HOST,,}_${now}.sql
632651
}
633652

653+
pre_dbbackup() {
654+
### Pre Script Support
655+
if [ -n "${PRE_SCRIPT}" ] ; then
656+
if var_true "${PRE_SCRIPT_SKIP_X_VERIFY}" ; then
657+
eval "${PRE_SCRIPT}" "${dbtype}" "${DB_HOST}" "${1}" "${dbbackup_start_time}" "${target}"
658+
else
659+
if [ -x "${PRE_SCRIPT}" ] ; then
660+
print_notice "Found PRE_SCRIPT environment variable. Executing '${PRE_SCRIPT}"
661+
eval "${PRE_SCRIPT}" "${dbtype}" "${DB_HOST}" "${1}" "${dbbackup_start_time}" "${target}"
662+
else
663+
print_error "Can't execute PRE_SCRIPT environment variable '${PRE_SCRIPT}' as its filesystem bit is not executible!"
664+
fi
665+
fi
666+
fi
667+
668+
### Pre Backup Custom Script Support
669+
if [ -d "/assets/custom-scripts/pre/" ] ; then
670+
for f in $(find /assets/custom-scripts/pre/ -name \*.sh -type f); do
671+
if var_true "${PRE_SCRIPT_SKIP_X_VERIFY}" ; then
672+
${f} "${dbtype}" "${DB_HOST}" "${1}" "${dbbackup_start_time}" "${target}"
673+
else
674+
if [ -x "${f}" ] ; then
675+
print_notice "Executing pre backup custom script : '${f}'"
676+
## script DB_TYPE DB_HOST DB_NAME STARTEPOCH BACKUP_FILENAME
677+
${f} "${dbtype}" "${DB_HOST}" "${1}" "${dbbackup_start_time}" "${target}"
678+
else
679+
print_error "Can't run pre backup custom script: '${f}' as its filesystem bit is not executible!"
680+
fi
681+
fi
682+
done
683+
fi
684+
}
685+
634686
post_dbbackup() {
635687
dbbackup_finish_time=$(date +"%s")
636688
dbbackup_total_time=$(echo $((dbbackup_finish_time-dbbackup_start_time)))

0 commit comments

Comments
 (0)