Skip to content

Commit cd348fb

Browse files
committed
postgresql: Add a script to upgrade PostgreSQL data.
This scripts the steps given in the upgrade documentation, altered for supporting Docker managed volumes.
1 parent 74aea02 commit cd348fb

File tree

2 files changed

+121
-62
lines changed

2 files changed

+121
-62
lines changed

README.md

Lines changed: 18 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -393,6 +393,13 @@ latest minor release within a major release series.
393393
(E.g. authentication settings for `memcached` became mandatory in
394394
the `2.1.2` release).
395395

396+
**Note:** Do not make any changes to the database version or
397+
volume. If there is a difference in database version, leave those
398+
unchanged for now, and complete that upgrade separately after the
399+
Zulip upgrade; see [the section below][pg-upgrade].
400+
401+
[pg-upgrade]: #upgrading-zulipzulip-postgresql-to-14
402+
396403
3. Verify that your updated `docker-compose.yml` points to the desired image version,
397404
e.g.:
398405
```yml
@@ -447,68 +454,17 @@ Then stop and restart the container as described in the previous section.
447454

448455
### Upgrading zulip/zulip-postgresql to 14
449456

450-
These instructions assume that you have not changed the default
451-
Postgres data path (`/opt/docker/zulip/postgresql/data`) in your
452-
`docker-compose.yml`. If you have changed it, please replace all
453-
occurences of `/opt/docker/zulip/postgresql/data` with your path.
454-
455-
1. Make a backup of your Zulip Postgres data dir.
456-
457-
2. Stop the Zulip container:
458-
459-
```
460-
docker-compose stop zulip
461-
```
462-
463-
3. Create a new (upgraded) Postgres container using a different data directory:
464-
465-
```
466-
docker run -d \
467-
--name postgresnew \
468-
-e POSTGRES_DB=zulip \
469-
-e POSTGRES_USER=zulip \
470-
-e POSTGRES_PASSWORD=zulip \
471-
-v /opt/docker/zulip/postgresql/new:/var/lib/postgresql/data:rw \
472-
zulip/zulip-postgresql:14
473-
```
474-
475-
4. Use `pg_dumpall` to dump all data from the existing Postgres container to the
476-
new Postgres container, and reset the password (for SCRAM-SHA-256 auth upgrade):
477-
478-
```
479-
docker-compose exec database pg_dumpall -U zulip | \
480-
docker exec -i postgresnew psql -U zulip
481-
482-
echo "ALTER USER zulip WITH PASSWORD 'REPLACE_WITH_SECURE_POSTGRES_PASSWORD';" |
483-
docker exec -i postgresnew psql -U zulip
484-
```
485-
486-
5. Stop and remove both Postgres containers:
487-
488-
```
489-
docker-compose rm --stop database
490-
docker stop postgresnew
491-
docker rm postgresnew
492-
```
493-
494-
6. Edit your `docker-compose.yml` to use the `zulip/zulip-postgresql:14` image
495-
for the `database` container.
496-
497-
7. Replace the old Postgres data directory with upgraded data directory:
498-
499-
```
500-
sudo mv /opt/docker/zulip/postgresql/data /opt/docker/zulip/postgresql/old
501-
sudo mv /opt/docker/zulip/postgresql/new /opt/docker/zulip/postgresql/data
502-
```
503-
504-
8. Start Zulip up again:
505-
506-
```
507-
docker-compose up
508-
```
509-
510-
That should be it. Your Postgres data has now been updated to use the
511-
`zulip/zulip-postgresql` image.
457+
The Docker Compose configuration for version 6.0-0 and higher default
458+
to using PostgreSQL 14, as the previously-used PostgreSQL 10 is no
459+
longer supported. Because the data is specific to the version of
460+
PostgreSQL which is running, it must be dumped and re-loaded into a
461+
new volume to upgrade. PostgreSQL 14 will refuse to start if provided
462+
with un-migrated data from PostgreSQL 10.
463+
464+
The provided `upgrade-postgresql` tool will dump the contents of the
465+
`postgresql` image's volume, create a new PostgreSQL 14 volume,
466+
perform the necessary migration, update the `docker-compose.yml`
467+
file to match, and re-start Zulip.
512468

513469
### Upgrading from the old galexrt/docker-zulip
514470

upgrade-postgresql

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
#!/usr/bin/env bash
2+
set -eux
3+
4+
new_version=14
5+
6+
# Require the `yq` tool
7+
if ! command -v yq >/dev/null; then
8+
echo "You must install the 'yq' tool to use this script."
9+
exit 1
10+
fi
11+
12+
image=$(yq ".services.database.image" docker-compose.yml)
13+
if [[ $image =~ ^zulip/zulip-postgresql:([0-9]+)$ ]]; then
14+
old_version="${BASH_REMATCH[1]}"
15+
else
16+
echo "Unexpected PostgreSQL image: $image"
17+
exit 1
18+
fi
19+
20+
volume_mount=$(yq ".services.database.volumes.0" docker-compose.yml)
21+
if [[ "$volume_mount" =~ ^([^:]+):/var/lib/postgresql/data:rw$ ]]; then
22+
old_mountpoint="${BASH_REMATCH[1]}"
23+
else
24+
echo "Unexpected volume mount: $volume_mount"
25+
exit 1
26+
fi
27+
28+
if [ "$new_version" -eq "$old_version" ]; then
29+
echo "PostgreSQL image is already version $new_version!"
30+
exit 1
31+
fi
32+
33+
# Create a new volume for the data; scope it with the current
34+
# directory, like docker-compose
35+
docker_compose_project="$(basename "$(pwd)")"
36+
new_volume="postgresql-$new_version"
37+
full_new_volume="${docker_compose_project}_${new_volume}"
38+
docker volume create "$full_new_volume"
39+
trap 'docker volume --force "$full_new_volume"' EXIT
40+
41+
# Start a new PostgreSQL container of the right version to read in the
42+
# dumped database and write a new data dir on the new volume
43+
temp_container=$(
44+
docker run -d \
45+
-e POSTGRES_DB=zulip \
46+
-e POSTGRES_USER=zulip \
47+
-e POSTGRES_PASSWORD=zulip \
48+
-v "$full_new_volume:/var/lib/postgresql/data:rw" \
49+
--health-cmd 'psql -U zulip -c "select 1"' \
50+
--health-interval 10s \
51+
"zulip/zulip-postgresql:$new_version"
52+
)
53+
trap 'docker volume rm --force "$full_new_volume"; docker rm --force "$temp_container"' EXIT
54+
55+
# Wait for the new PostgreSQL container to become available
56+
tries=0
57+
while [ "$(docker inspect --format='{{json .State.Health.Status}}' "$temp_container")" != '"healthy"' ]; do
58+
tries=$((tries + 1))
59+
if [ "$tries" -gt 5 ]; then
60+
echo "PostgreSQL $new_version container failed to start!"
61+
exit 1
62+
fi
63+
sleep 10
64+
done
65+
66+
# Ensure database is running
67+
docker-compose up --wait database
68+
69+
# Stop the zulip processes which talk to the database
70+
zulip_is_running=$(docker-compose ps --filter status=running --services | grep zulip || true)
71+
if [ -n "$zulip_is_running" ]; then
72+
docker-compose stop zulip
73+
fi
74+
75+
# Transfer the data to the new container
76+
docker-compose exec database pg_dumpall -U zulip |
77+
docker exec -i "$temp_container" psql -U zulip
78+
79+
if [ "$old_version" -eq "10" ]; then
80+
# Upgrade MD5 password to SCRAM-SHA-256. We escape all 's by doubling them.
81+
database_password=$(yq .services.database.environment.POSTGRES_PASSWORD docker-compose.yml |
82+
perl -pe "s/'/''/g")
83+
echo "ALTER USER zulip WITH PASSWORD '$database_password';" |
84+
docker exec -i "$temp_container" psql -U zulip
85+
fi
86+
87+
# Stop the running database
88+
docker-compose rm --force --stop database
89+
90+
# Stop the temporary PostgreSQL container
91+
docker stop "$temp_container"
92+
docker rm "$temp_container"
93+
trap '' EXIT
94+
95+
# Update the docker-compose.yml file for the new version and data path
96+
IMAGE="zulip/zulip-postgresql:$new_version" yq -i '.services.database.image = strenv(IMAGE)' docker-compose.yml
97+
VOLUME="$new_volume:/var/lib/postgresql/data:rw" yq -i '.services.database.volumes.0 = strenv(VOLUME)' docker-compose.yml
98+
VOLUME="$new_volume" yq -i 'with(.volumes.[strenv(VOLUME)]; . = "" | . tag="!!null")' docker-compose.yml
99+
100+
# Restart zulip, and the new database container
101+
docker-compose up --wait
102+
103+
echo "Old data from PostgreSQL $old_version is available in $old_mountpoint"

0 commit comments

Comments
 (0)