Skip to content

Commit 0927788

Browse files
jsilvelajsilvela
authored
docs: new blog post on using Atlas (#267)
Signed-off-by: Jaime Silvela <[email protected]>
1 parent afee0d3 commit 0927788

File tree

3 files changed

+210
-3
lines changed

3 files changed

+210
-3
lines changed

‎content/authors/jsilvela/index.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,14 @@ avatar: jsilvela-gh.jpeg
44
github: jsilvela
55
---
66

7-
Jaime Silvela has been around. In a career spanning two decades writing code,
7+
Jaime Silvela has been around. In a long and varied career writing code,
88
he has lived in the US and Europe, worked for big corporations like Amazon and
99
Microsoft, as an early startup employee, in industries including Wall Street and
1010
catastrophe insurance.
1111

1212
Early in his career he discovered a fondness for databases, PostgreSQL in
13-
particular. He started using Go professionally in 2014, after having suffered
14-
Java.
13+
particular. He started using Go professionally in 2014, a relief after having
14+
suffered the Java monoculture at Amazon.
1515

1616
Jaime joined EDB in 2021, and he is very proud to be able to put his skills to
1717
use on Kubernetes operators for databases, including CloudNativePG for PostgreSQL

‎content/blog/migrations-with-atlas-and-cloudnativepg/elephant-migration.jpg‎

183 KB
Loading

‎content/blog/migrations-with-atlas-and-cloudnativepg/index.md‎

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
---
2+
title: "Migrations With Atlas and CloudNativePG"
3+
date: 2025-05-14T22:34:18+02:00
4+
draft: false
5+
image:
6+
url: elephant-migration.jpg
7+
attribution: from <strong><a href="https://commons.wikimedia.org/w/index.php?curid=101232202">Wikimedia Commons</a></strong>
8+
author: jsilvela
9+
tags:
10+
- blog
11+
- tutorial
12+
- migrations
13+
- devops
14+
summary: Doing schema migrations on CloudNativePG clusters using the Atlas operator
15+
---
16+
17+
One of the most important practices when developing code that relies on
18+
databases is to use *database migration tools* for change management.
19+
It's something you *will* learn, even if it has to be the hard way.
20+
(Another thing I see too many newcomers learning the hard way is to take backups
21+
often, and to test those backups regularly.)
22+
23+
In the post [*Developing webapps with CloudNativePG*]({{% ref "/blog/developing-webapps-with-cloudnative-pg" %}}),
24+
we mentioned [Liquibase](https://www.liquibase.com), which is one of the most
25+
popular database migration tools.
26+
27+
Traditional database migration tools assume a connection is available to the
28+
target database. In the context of Kubernetes,
29+
and of Postgresql clusters built using CloudNativePG, to use such a tool
30+
we would need to expose the database *service* outside the Kubernetes cluster,
31+
for example via
32+
[port forwarding](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/),
33+
an [ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/),
34+
or similar solutions.
35+
36+
The [Atlas database migration tool](https://atlasgo.io) includes a Kubernetes
37+
operator that lets us manage database migrations within Kubernetes.
38+
Let's see an example of how to do that:
39+
40+
## Step 0: install CloudNativePG and create a Postgres cluster
41+
42+
First of all, you should have CloudNativePG running on your kubernetes cluster,
43+
and a Postgres cluster created with CloudNativePG.
44+
If you don't yet have this, you can follow the
45+
[CloudNativePG quickstart](https://cloudnative-pg.io/documentation/current/quickstart/).
46+
47+
Whether you follow the quickstart or you already had a CloudNativePG/Postgres
48+
cluster up and running, we'll assume for the rest of this post that your
49+
CloudNativePG cluster is called `cluster-example`.
50+
51+
## Step 1: install the Atlas operator
52+
53+
To install the Atlas operator, you can use Helm:
54+
55+
``` console
56+
helm install atlas-operator oci://ghcr.io/ariga/charts/atlas-operator
57+
```
58+
59+
Note that you may need an access token to retrieve the image from
60+
the GHCR registry (please see the [GHCR documentation](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-to-the-container-registry)
61+
for details).
62+
63+
After Atlas is installed, you will notice new CRDs in your Kubernetes
64+
installation:
65+
66+
``` console
67+
> kubectl get crd | grep atlas
68+
atlasmigrations.db.atlasgo.io <timestamp>
69+
atlasschemas.db.atlasgo.io <timestamp>
70+
```
71+
72+
## Step 2: make changes to the database using Atlas
73+
74+
We will use the *Atlas Schema* CRD to manage migrations. You may want
75+
to open the [Atlas operator quickstart](https://atlasgo.io/integrations/kubernetes/quickstart)
76+
for reference, though we're not going to follow it exactly.
77+
78+
To apply a migration, we need credentials to access our target database.
79+
The Atlas operator quickstart uses the `urlFrom`
80+
stanza, but with CloudNativePG there is a more convenient way.
81+
82+
From Step 0, we have assumed we have a CloudNativePG cluster called
83+
`cluster-example`.
84+
CloudNativePG, by default, creates a database called `app` on clusters, and
85+
a user `app` whose credentials are held in a Secret called
86+
`cluster-example-app`:
87+
88+
``` console
89+
> kubectl get secrets | grep app
90+
cluster-example-app kubernetes.io/basic-auth 9 18h
91+
```
92+
93+
You can inspect the contents of the secret running
94+
`kubectl get secrets cluster-example-app -o yaml`,
95+
and you will find that it contains a key called `password`, holding of course
96+
the password for the `app` user (base64 encoded).
97+
In addition to the `cluster-example-app` Secret, the CloudNativePG operator
98+
creates Services for Postgres. In particular, we will want to use the ReadWrite
99+
service, called `cluster-example-rw`, for the migrations.
100+
101+
We're going to use the [`credentials` object](https://atlasgo.io/integrations/kubernetes/declarative#credentials-object)
102+
from the AtlasSchema CRD to reference
103+
the password and the service. Following along the Atlas Operator Quickstart, we
104+
create a migration defining a table called `t1`. Save the following to a file
105+
named `atlas-schema.yaml`.
106+
Notice how CloudNativePG automatically produced the values we need to put
107+
in the `passwordFrom` and `host` fields:
108+
109+
``` yaml
110+
apiVersion: db.atlasgo.io/v1alpha1
111+
kind: AtlasSchema
112+
metadata:
113+
name: atlasschema-pg
114+
spec:
115+
credentials:
116+
scheme: postgres
117+
host: cluster-example-rw.default
118+
user: app
119+
passwordFrom:
120+
secretKeyRef:
121+
key: password
122+
name: cluster-example-app
123+
database: app
124+
port: 5432
125+
parameters:
126+
sslmode: disable
127+
schema:
128+
sql: |
129+
create table t1 (
130+
id int
131+
);
132+
```
133+
134+
Then apply it: `kubectl apply -f atlas-schema.yaml`.
135+
136+
In a few seconds, you should be able to see the Schema has been reconciled:
137+
138+
``` console
139+
> kubectl get atlasschemas.db.atlasgo.io
140+
NAME READY REASON
141+
atlasschema-pg True Applied
142+
```
143+
144+
To verify the effects on the database, let's get a `psql` session open on one
145+
of our instances:
146+
147+
``` console
148+
> kubectl exec -ti cluster-example-1 -- psql app
149+
Defaulted container "postgres" out of: postgres, bootstrap-controller (init)
150+
psql (17.4 (Debian 17.4-1.pgdg110+2))
151+
Type "help" for help.
152+
153+
app=# \dt
154+
List of relations
155+
Schema | Name | Type | Owner
156+
--------+------+-------+-------
157+
public | t1 | table | app
158+
(1 row)
159+
160+
app=# \d t1
161+
Table "public.t1"
162+
Column | Type | Collation | Nullable | Default
163+
--------+---------+-----------+----------+---------
164+
id | integer | | |
165+
```
166+
167+
As suggested in the Atlas Quickstart, we can modify the schema in the
168+
`atlas-schema.yaml` file, and then re-apply with
169+
`kubectl apply -f atlas-schema.yaml`, and the Atlas operator will again
170+
reconcile the database to the desired state.
171+
172+
## What's the point? DevOps
173+
174+
Let's say you have an application that uses a database. In production, the
175+
database may hold your customers's data, purchase history, financial
176+
transactions, perhaps maps using PostGIS, etc.
177+
178+
Application developers will regularly need to add functionality, and often
179+
that will involve creating new tables, schemas, procedures, indexes, perhaps
180+
doing some `INSERT` statements to populate static data, etc.
181+
The developers will be using a development database, probably hosted in their
182+
dev machines.
183+
There might be another database for the purpose of automated testing, and
184+
of course there's the production database, where the changes will need to be
185+
applied in the end. These different databases will have different data in them,
186+
and different loads.
187+
Add time and other developers, and you have a big chance for snafus of the type
188+
*"but it worked on my machine!"*
189+
190+
Database migration tools help manage this, bringing DevOps to this area of
191+
databases. Atlas, by working as a Kubernetes operator, makes the dev/prod
192+
transition even smoother. \
193+
But it's not enough to use good tools. Remember the Agile manifesto:
194+
*"individuals and interactions over processes and tools"*.
195+
You can find an overview of the challenges posed by database change management
196+
at [DORA](https://dora.dev/capabilities/database-change-management/).
197+
198+
Using CloudNativePG, with its services and credentials secrets created out of
199+
the box, together with Atlas, will enable the developers to create migrations
200+
in their development Kubernetes installations, update the YAML
201+
files for Atlas in version control to share with the other devs, and eventually
202+
apply to the production database smoothly.
203+
204+
---
205+
206+
This intro was just to whet your appetite. There is plenty more to learn with
207+
Atlas.

0 commit comments

Comments
 (0)