Skip to content

Commit 1c22bfe

Browse files
gbartolinigbartolini
committed
feat: CNPG Recipe 15 - declarative logical replication upgrades
Signed-off-by: Gabriele Bartolini <[email protected]>
1 parent 5a3cf42 commit 1c22bfe

File tree

8 files changed

+341
-0
lines changed

8 files changed

+341
-0
lines changed

‎content/posts/20241210-major-online-upgrades/cover.jpg‎

330 KB
Loading

‎content/posts/20241210-major-online-upgrades/index.md‎

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
---
2+
title: "CNPG Recipe 15 - PostgreSQL major online upgrades with logical replication"
3+
date: 2024-12-11T07:28:42+01:00
4+
description: ""
5+
tags: ["postgresql", "postgres", "kubernetes", "k8s", "cloudnativepg", "cnpg", "postgresql", "postgres", "dok", "data on kubernetes"]
6+
cover: cover.jpg
7+
thumb: thumb.jpg
8+
draft: false
9+
---
10+
11+
_This recipe shows how to perform an online major PostgreSQL upgrade using the
12+
new declarative approach to logical replication introduced in CloudNativePG
13+
1.25. By leveraging the `Publication` and `Subscription` CRDs, users can set up
14+
logical replication between PostgreSQL clusters with ease. I will walk you
15+
through configuring a PostgreSQL 15 publisher, importing schemas into a
16+
PostgreSQL 17 subscriber, and verifying data synchronisation, with the broader
17+
goal of highlighting the benefits of a repeatable and testable upgrade process._
18+
19+
<!--more-->
20+
21+
---
22+
23+
Upgrading PostgreSQL to a major version has historically been one of the most
24+
challenging tasks for users. To minimise downtime during the upgrade process,
25+
two primary approaches are commonly used:
26+
27+
1. **Offline In-Place Upgrade**
28+
Using [`pg_upgrade`](https://www.postgresql.org/docs/current/pgupgrade.html)
29+
to ensure that the data files from an older version are converted to the newer
30+
version.
31+
32+
2. **Online Remote Upgrade**
33+
Leveraging native logical replication for minimal downtime. For a detailed
34+
guide, check out ["CNPG #5 - How to Migrate Your PostgreSQL Database in Kubernetes with ~0 Downtime from Anywhere"]({{< relref "../20240327-zero-cutover-migrations/index.md" >}}),
35+
as it covers concepts essential for fully understanding this article.
36+
37+
38+
Although CloudNativePG does not currently support `pg_upgrade`
39+
([planned for version 1.26](https://github.com/cloudnative-pg/cloudnative-pg/issues/4682)),
40+
version 1.25 introduces a significant new feature:
41+
[declarative support for logical publications and subscriptions](https://cloudnative-pg.io/documentation/preview/logical_replication/).
42+
43+
This article demonstrates how to perform an online major PostgreSQL upgrade on
44+
your laptop using `kind` (Kubernetes in Docker). I'll use the `Publication` and
45+
`Subscription` CRDs introduced in the **preview** version `1.25.0-rc1`.
46+
47+
Please help us test this release candidate in a non-production environment by
48+
following the [instructions provided here](https://cloudnative-pg.io/documentation/preview/preview_version/).
49+
50+
## Use Case: Online Major PostgreSQL Upgrade
51+
52+
Imagine you have a CloudNativePG-managed PostgreSQL 15 database running in your
53+
Kubernetes cluster. Your goal is to upgrade it to PostgreSQL 17 while meeting
54+
the following requirements:
55+
56+
- *Repeatability*: The process should be repeatable to ensure reliability.
57+
- *Testability*: Run a test migration to provide ample time for developers
58+
and testers to validate the upgraded database. This also allows you to
59+
measure the time required for data transfer, and practice the cutover
60+
procedure.
61+
- *Production-Readiness*: Ensure a fresh migration is performed from scratch
62+
before switching to the new database in production. This approach minimises
63+
downtime for the application, reducing it to near-zero levels.
64+
65+
## Installing CloudNativePG Preview Version 1.25.0
66+
67+
Before proceeding, ensure you’ve completed the steps in
68+
["CloudNativePG Recipe 1 - Setting Up Your Local Playground in Minutes"]({{< relref "../20240303-recipe-local-setup/index.md" >}}).
69+
For this example, you’ll need version 1.25 to use the declarative `Publication`
70+
and `Subscription` CRDs.
71+
72+
To install version 1.25.0-rc1, run the following command:
73+
74+
```sh
75+
kubectl apply --server-side -f \
76+
https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/main/releases/cnpg-1.25.0-rc1.yaml
77+
```
78+
79+
Make sure you also have the latest version of the `cnpg` plugin for `kubectl`,
80+
as described in CloudNativePG Recipe #1.
81+
82+
## Setting Up the Origin Database
83+
84+
Let’s start by defining a PostgreSQL 15 cluster with the following
85+
configuration:
86+
87+
```yaml
88+
{{< include "yaml/pg-15.yaml" >}}
89+
```
90+
91+
### Populating the Database with Sample Data
92+
93+
We’ll use `pgbench` to generate and insert sample data into the database. Run
94+
the following command:
95+
96+
```sh
97+
kubectl cnpg pgbench \
98+
--db-name app \
99+
--job-name pgbench-init \
100+
pg-15 \
101+
-- --initialize
102+
```
103+
104+
After running the command, inspect the job logs to confirm the operation:
105+
106+
```sh
107+
kubectl logs jobs/pgbench-init
108+
```
109+
110+
You should see output similar to the following:
111+
112+
```output
113+
dropping old tables...
114+
NOTICE: table "pgbench_accounts" does not exist, skipping
115+
NOTICE: table "pgbench_branches" does not exist, skipping
116+
NOTICE: table "pgbench_history" does not exist, skipping
117+
NOTICE: table "pgbench_tellers" does not exist, skipping
118+
creating tables...
119+
generating data (client-side)...
120+
100000 of 100000 tuples (100%) of pgbench_accounts done (elapsed 0.01 s, remaining 0.00 s)
121+
vacuuming...
122+
creating primary keys...
123+
done in 0.16 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 0.07 s, vacuum 0.04 s, primary keys 0.04 s).
124+
```
125+
126+
At this point, the `app` database should contain the `pgbench` tables and be
127+
approximately 22 MB in size. You can verify this by running:
128+
129+
```sh
130+
kubectl cnpg psql pg-15 -- app -c '\dt+' -c '\l+'
131+
```
132+
133+
## Setting Up the Destination Database
134+
135+
Next, you’ll define the PostgreSQL 17 cluster and configure it to import the
136+
schema from its source, `pg-15`, which serves as the "publisher" in logical
137+
replication terms:
138+
139+
```yaml
140+
{{< include "yaml/pg-17.yaml" >}}
141+
```
142+
143+
In the example above, the `externalClusters` section specifies how to connect
144+
to the publisher using the `app` user.
145+
146+
Once the manifest is applied, the destination server will have the schema
147+
imported but without any data. You can verify this by running:
148+
149+
```sh
150+
kubectl cnpg psql pg-17 -- app -c '\dt+'
151+
```
152+
153+
You should get a similar output:
154+
155+
```output
156+
List of relations
157+
Schema | Name | Type | Owner | Persistence | Access method | Size | Description
158+
--------+------------------+-------+-------+-------------+---------------+---------+-------------
159+
public | pgbench_accounts | table | app | permanent | heap | 0 bytes |
160+
public | pgbench_branches | table | app | permanent | heap | 0 bytes |
161+
public | pgbench_history | table | app | permanent | heap | 0 bytes |
162+
public | pgbench_tellers | table | app | permanent | heap | 0 bytes |
163+
(4 rows)
164+
```
165+
166+
## Setting Up the Publisher
167+
168+
The first step in configuring logical replication is enabling a role to be used
169+
in a logical publication in the `app` database. For this example, we’ll use the
170+
`app` user for simplicity. Below, we assign the `replication` privilege to the
171+
`app` user:
172+
173+
```yaml
174+
{{< include "yaml/pg-15-pub.yaml" >}}
175+
```
176+
177+
We then create the `Publication` resource that will be then used to replicate
178+
all changes to all tables in the `app` database.
179+
180+
```yaml
181+
{{< include "yaml/pub.yaml" >}}
182+
```
183+
184+
Apply the manifest and check the `app` user:
185+
186+
```sh
187+
kubectl cnpg psql pg-15 -- app -c '\du app'
188+
```
189+
```output
190+
List of roles
191+
Role name | Attributes
192+
-----------+-------------
193+
app | Replication
194+
```
195+
196+
## Setting Up the Subscriber
197+
198+
With the `pg-17` database schema imported, the next step is to configure the
199+
subscription to the "publisher" on `pg-15`. This establishes logical
200+
replication between the two clusters:
201+
202+
```yaml
203+
{{< include "yaml/sub.yaml" >}}
204+
```
205+
206+
Apply the manifest to initiate the subscription, then check the logs of both
207+
clusters. You should see entries indicating that logical replication has
208+
started, such as:
209+
210+
```
211+
logical replication apply worker for subscription "subscriber" has started
212+
```
213+
214+
or:
215+
216+
```
217+
starting logical decoding for slot "subscriber"
218+
```
219+
220+
Finally, verify that the `pgbench_accounts` table on the destination database
221+
contains 10,000 records by running the following command:
222+
223+
```sh
224+
kubectl cnpg psql pg-17 -- app -qAt -c 'SELECT count(*) FROM pgbench_accounts'
225+
```
226+
227+
Once you have migrated, you can delete the subscription and publication
228+
objects.
229+
230+
## Conclusions
231+
232+
This article introduced the basics of setting up logical replication using the
233+
new declarative approach in CloudNativePG 1.25. While the focus was on core
234+
concepts, there are additional aspects to consider, such as synchronising
235+
sequences—which are not part of the `pgbench` database. Detailed guidance on
236+
this topic is available in [CNPG Recipe #5](https://www.gabrielebartolini.it/articles/2024/03/cloudnativepg-recipe-5-how-to-migrate-your-postgresql-database-in-kubernetes-with-~0-downtime-from-anywhere/)
237+
through the use of plugins— this task is still imperative or delegated to the
238+
application.
239+
240+
For cutover, transitioning your applications to the new `pg-17-rw` service will
241+
finalise the process. If your publisher resides outside the control of
242+
CloudNativePG, you can still benefit from the declarative subscriptions.
243+
244+
The primary goal of this article was to showcase the streamlined and repeatable
245+
process for managing PostgreSQL native logical replication through the new
246+
declarative method for defining publications and subscriptions. This approach
247+
significantly enhances the manageability and efficiency of logical replication
248+
in Kubernetes environments.
249+
250+
---
251+
252+
Stay tuned for the upcoming recipes! For the latest updates, consider
253+
subscribing to my [LinkedIn](https://www.linkedin.com/in/gbartolini/) and
254+
[Twitter](https://twitter.com/_GBartolini_) channels.
255+
256+
If you found this article informative, feel free to share it within your
257+
network on social media using the provided links below. Your support is
258+
immensely appreciated!
259+
260+
_Cover Picture: [“Elephant Family with baby“](https://www.flickr.com/photos/barbourians/6168258059)._
261+

‎content/posts/20241210-major-online-upgrades/thumb.jpg‎

130 KB
Loading

‎content/posts/20241210-major-online-upgrades/yaml/pg-15-pub.yaml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
apiVersion: postgresql.cnpg.io/v1
2+
kind: Cluster
3+
metadata:
4+
name: pg-15
5+
spec:
6+
instances: 3
7+
8+
storage:
9+
size: 1Gi
10+
11+
managed:
12+
roles:
13+
- name: app
14+
login: true
15+
replication: true

‎content/posts/20241210-major-online-upgrades/yaml/pg-15.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
apiVersion: postgresql.cnpg.io/v1
2+
kind: Cluster
3+
metadata:
4+
name: pg-15
5+
spec:
6+
instances: 3
7+
8+
storage:
9+
size: 1Gi

‎content/posts/20241210-major-online-upgrades/yaml/pg-17.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
apiVersion: postgresql.cnpg.io/v1
2+
kind: Cluster
3+
metadata:
4+
name: pg-17
5+
spec:
6+
instances: 3
7+
8+
storage:
9+
size: 1Gi
10+
11+
# Use the microservice import for the schema
12+
bootstrap:
13+
initdb:
14+
import:
15+
type: microservice
16+
schemaOnly: true
17+
databases:
18+
- app
19+
source:
20+
externalCluster: pg-15
21+
22+
# Define the "publisher"
23+
externalClusters:
24+
- name: pg-15
25+
connectionParameters:
26+
host: pg-15-rw.default.svc
27+
user: app
28+
dbname: app
29+
password:
30+
name: pg-15-app
31+
key: password

‎content/posts/20241210-major-online-upgrades/yaml/pub.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
apiVersion: postgresql.cnpg.io/v1
2+
kind: Publication
3+
metadata:
4+
name: pg-15-publisher
5+
spec:
6+
cluster:
7+
name: pg-15
8+
dbname: app
9+
name: publisher
10+
target:
11+
allTables: true
12+
publicationReclaimPolicy: delete

‎content/posts/20241210-major-online-upgrades/yaml/sub.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: postgresql.cnpg.io/v1
2+
kind: Subscription
3+
metadata:
4+
name: pg-17-subscription
5+
spec:
6+
cluster:
7+
name: pg-17
8+
dbname: app
9+
name: subscriber
10+
externalClusterName: pg-15
11+
publicationName: publisher
12+
subscriptionReclaimPolicy: delete
13+

0 commit comments

Comments
 (0)