Server Backup/Restore

[JeremyClement]

It would be nice to be able to backup the server with the database and being able to restore it incase of a disk crash or server migration.

[alextran1502]

This should happen in the user space, similar to how you would back up other applications.

[tomislav177]

I also have questions about how to perform backup and restore after reinstalling rpi4 for example. If it is done by duplicati which folder to backup? I think that this is very important because I have my photo on external HDD if I move external HDD with folder structure that was created by Immich to another install of Immich. I just want to restore the database with folder structure and list of users but how to do that?
is it enough with duplicati backup:
immich-app_pgdata volume that was created by docker?

[jpmiller25]

@tomislav177 @JeremyClement
I run a daily and weekly cron job to backup my pgdata volume into my immich directory (alongside my docker-compose.yaml and .env), then duplicati is pointed to that directory so it backs up the backup to AWS.
I'm not experienced at scripting so it may not be very efficient, but seems to get the job done:
Crontab:

15 1 * * * /home/josh/scripts/daily-bkup.sh
0  1 * * 1 /home/josh/scripts/weekly-bkup.sh

daily-bkup.sh

#!/bin/sh

## Daily Immich DB Backup
## Docker volume backup re:  https://docs.docker.com/storage/volumes/#back-up-a-volume

cd /home/josh/immich
docker compose down

cd /home/josh/immich/db-backups
docker run --rm --volumes-from immich_postgres -v $(pwd):/backup ubuntu tar cvf /backup/daily.tar /var/lib/postgresql/data

cd /home/josh/immich
docker compose up -d

weekly-bkup.sh:

#!/bin/sh

## Weekly Immich DB Backup
## Docker volume backup re:  https://docs.docker.com/storage/volumes/#back-up-a-volume

cd /home/josh/immich/db-backups
mv -f daily.tar weekly.tar

This gets me a daily backup, and a weekly in case the daily fails at some point. I think it's enough redundancy for me.

[nep2ner]

Has anyone gotten a successful restore using this method?

With backup command:
docker run --rm --volumes-from immich_postgres -v /host/machine/db_backups:/db_backups ubuntu tar cvf /db_backups/backup.tar /var/lib/postgresql/data

The command below imports the data back into the immich_pgdata volume (at /var/lib/postgressql/data), but Immich web still shows the set up admin user screen after bringing the containers back up.

docker run --rm --volumes-from immich_postgres -v /host/machine/db_backups:/db_backups ubuntu bash -c "tar xvf /db_backups/backup.tar"

Is there anything else I need to do?

[bo0tzz]

The last command you posted doesn't unpack the data into /var/lib/postgresql/data, but into whatever the current working directory is.

[nep2ner]

The contents of backup.tar look like this: /var/lib/postgresql/data/<content>

So I think the data restores to the right place.

Also by mounting the volume to a standard ubuntu container, I can see that /var/lib/postgresql/data/ does not exist before running that last command, but does exist after running it.

I'm a docker noob, but it does seem like the data is getting to the right place.

[nep2ner]

Could it be that backing up /var/lib/postgresql/data/ isn't sufficient? Is there more important data in the postgres db volume than that?

[jrasm91]

At the end of the day there are at least two ways to backup a postgres database, as mentioned in the docs filesystem vs sql dump.

File storage has some caveats, I would probably recommend using pgdump. Something like this: docker exec -t immich_postgres pg_dumpall -c -U postgres | gzip > "/volume1/docker/immich/dump.sql.gz"

https://www.postgresql.org/docs/current/backup.html

[nep2ner]

Looks like there are some great official instructions on this topic now! I'm trying it out now.

https://documentation.immich.app/docs/administration/backup-and-restore

Thanks @jrasm91!

[nep2ner]

I get a lot of errors like these (there were many like each line shown below but with different targets)

ERROR:  relation "albums" already exists
ERROR:  insert or update on table "albums" violates foreign key constraint 
DETAIL:  Key is not present in table "assets".
ERROR:  duplicate key value violates unique constraint "PK_"
DETAIL:  Key (id)=(1) already exists.
ERROR:  multiple primary keys for table "user_token" are not allowed
ERROR:  constraint "FK_f8e8a9e893cb5c54907f1b798e9" for relation "tag_asset" already exists

when using gunzip < db_dumps/last/immich-latest.sql.gz | sudo docker exec -i immich_postgres psql -U postgres -d immich to restore. Is this expected?

[bo0tzz]

Are you restoring into an empty database?

[nep2ner]

I had to run docker compose up -d again to create the immich_postgress container that the command uses (otherwise that restore command fails). I think that the compose command puts some stuff into the database. But I found that if I run these two commands before the restore command, everything seems to go smoothly:

 sudo docker exec -i immich_postgres dropdb immich -U postgres
 sudo docker exec -i immich_postgres createdb immich -U postgres

[jrasm91]

The db restore process also restores the schema, which would result in some errors if it already existed. Dropping the database prior to restoring seems like a good idea.

[Vendo232]

@nep2ner this is GOLD, thank you it helpe me

sudo docker exec -i immich_postgres dropdb immich -U postgres
sudo docker exec -i immich_postgres createdb immich -U postgres

PLEASE ADD THIS TO THE GUIDE !

[VerTiGoEtrex]

How robust is immich when restoring? The documentation only discusses backing up the postgres db, but what about all of the generated files?

Off the top of my head:

• original photos themselves (obviously should back this up)
• postgres db data (obviously should back this up)
• transcoded videos (I don't want to waste space backing these up, but if I restore postgres and the originals without the transcodes, will immich gracefully regenerate the transcodes for me?)
• thumbnails (ditto - don't want to back these up, but will immich choke and die if it expects to find them but they're not there anymore?)
• typesense data (tbh, not really sure what's stored in here. I'm guessing face data? I have this bind-mounted rather than in a docker volume per the default docker-compose so it's easy for me to back up, but I'm not sure if that's desirable/necessary or not)

[bo0tzz]

Only the original photos and the postgres db have to be backed up. Immich will have trouble with missing transcodes and thumbnails, but running the jobs to regenerate all of them after a restore should take care of that. Typesense's data is generated based on the postgres DB automatically, so that doesn't need any manual restoring.

[VerTiGoEtrex]

Great, thanks!

[ozkyle]

What I did for backup and restore is like:

1. Edit the docker-compose.yml file, change this entry from "- pgdata:" to "- /opt/immich-app/pgdata:" in container named database:
   volumes:
   • /opt/immich-app/pgdata:/var/lib/postgresql/data

Or use this command to apply the change: sed -i "s#- pgdata#- /opt/immich-app/pgdata#g" ./docker-compose.yml

/opt/immich-app/pgdata is where your database files are located, and the whole directory can be copied and backed up to anywhere and can be used again as a restore action when you delete all the containers and volumes and restart a new immich server

2. Comment the pgdata volume, since this volume is not needed any more: volumes:
   volumes:
   #pgdata:

3. If you want to migrate the previous database folder/volume, just copy it to the new directory as below and recreate containers:
   sudo cp -rp /var/lib/docker/volumes/immich-app_pgdata/_data /opt/immich-app/pgdata/ or if run docker as non-root user,
   sudo cp -rp /home/user/.local/share/docker/volumes/immich-app_pgdata/_data /opt/immich-app/pgdata/

BTW, just a suggestion that it could be an option in the .env file that user can config the pgdata volume as database file location or use a custom location for it.