|
| 1 | +--- |
| 2 | +title: "Migrations With Atlas and CloudNativePG" |
| 3 | +date: 2025-03-19T11:47:56+01:00 |
| 4 | +draft: false |
| 5 | +image: |
| 6 | + url: elephant-migration.jpg |
| 7 | + attribution: from <strong><a href="https://commons.wikimedia.org/w/index.php?curid=101232202">Wikimedia Commons</a></strong> |
| 8 | +author: jsilvela |
| 9 | +tags: |
| 10 | + - blog |
| 11 | + - migrations |
| 12 | + - devops |
| 13 | +summary: Doing schema migrations on CloudNativePG clusters using the Atlas operator |
| 14 | +--- |
| 15 | + |
| 16 | +One of the most important practices when developing code that relies on |
| 17 | +databases is to use *database migration tools* for change management. |
| 18 | +It's something you *will* learn, even if it has to be the hard way. I hope |
| 19 | +you didn't have to learn it the hard way though! |
| 20 | +(Another thing I see too many newcomers learning the hard way is to take backups |
| 21 | +often, and to test those backups with some regularity.) |
| 22 | + |
| 23 | +In the post [*Developing webapps with CloudNativePG*]({{% ref "/blog/developing-webapps-with-cloudnative-pg" %}}), |
| 24 | +we mentioned [Liquibase](https://www.liquibase.com), which is one of the best |
| 25 | +known database migration tools. |
| 26 | + |
| 27 | +Traditional database migration tools assume a connection is available to the |
| 28 | +database one wants to perform migrations on. In the context of Kubernetes, |
| 29 | +and of Postgresql clusters built using CloudNativePG, |
| 30 | +we would need to expose the database service beyond the Kubernetes cluster, |
| 31 | +for example via |
| 32 | +[port forwarding](https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/), |
| 33 | +an [ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), |
| 34 | +or similar solutions. |
| 35 | + |
| 36 | +The [Atlas database migration tool](https://atlasgo.io) includes a Kubernetes |
| 37 | +operator that allows us to manage database migrations in a Kubernetes-native |
| 38 | +way. |
| 39 | +Let's see an example of how to do that: |
| 40 | + |
| 41 | +## Step 0: install CloudNativePG and create a Postgres cluster |
| 42 | + |
| 43 | +First of all, you should have CloudNativePG running on your kubernetes cluster, |
| 44 | +and a Postgres cluster created with CloudNativePG. |
| 45 | +If you don't yet have this, you can follow the |
| 46 | +[CloudNativePG quickstart](https://cloudnative-pg.io/documentation/current/quickstart/). |
| 47 | + |
| 48 | +Whether you follow the quickstart or you already had a CloudNativePG/Postgres |
| 49 | +cluster up and running, let's assume your CloudNativePG cluster is called |
| 50 | +`cluster-example`. |
| 51 | + |
| 52 | +## Step 1: install the Atlas operator |
| 53 | + |
| 54 | +To install the Atlas operator, you can use Helm: |
| 55 | + |
| 56 | +``` console |
| 57 | +helm install atlas-operator oci://ghcr.io/ariga/charts/atlas-operator |
| 58 | +``` |
| 59 | + |
| 60 | +Note that you may need an access token to retrieve the image from |
| 61 | +the GHCR registry (please see the [documentation](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-to-the-container-registry) |
| 62 | +for details). |
| 63 | + |
| 64 | +After Atlas is installed, you will notice new CRDs in your Kubernetes |
| 65 | +installation: |
| 66 | + |
| 67 | +``` console |
| 68 | +> kubectl get crd | grep atlas |
| 69 | +atlasmigrations.db.atlasgo.io 2025-03-18T16:49:08Z |
| 70 | +atlasschemas.db.atlasgo.io 2025-03-18T16:49:08Z |
| 71 | +``` |
| 72 | + |
| 73 | +## Step 2: make changes to the database using Atlas |
| 74 | + |
| 75 | +We will use the *Atlas Schema* CRD to manage migrations. You may want |
| 76 | +to open the [Atlas operator quickstart](https://atlasgo.io/integrations/kubernetes/quickstart) |
| 77 | +for reference, though we're not going to follow it exactly. |
| 78 | + |
| 79 | +To apply a migration, we connect to our target database, for which |
| 80 | +we need credentials. The Atlas operator quickstart uses the `urlFrom` |
| 81 | +stanza, but with CloudNativePG there is a more convenient way. |
| 82 | + |
| 83 | +From Step 0, we assumed we have a CloudNativePG cluster called |
| 84 | +`cluster-example`. |
| 85 | +CloudNativePG, by default, creates a database called `app` on the cluster, and |
| 86 | +a user `app` whose credentials are held in a Secret called |
| 87 | +`cluster-example-app`: |
| 88 | + |
| 89 | +``` console |
| 90 | +> kubectl get secrets | grep app |
| 91 | +cluster-example-app kubernetes.io/basic-auth 9 18h |
| 92 | +``` |
| 93 | + |
| 94 | +You may inspect the contents of the secret running `kubectl get secrets cluster-example-app -o yaml`, |
| 95 | +and you will find that it contains a key called `password`, holding of course |
| 96 | +the password for the `app` user (base64 encoded). |
| 97 | +In addition to the `cluster-example-app` Secret, the CloudNativePG operator |
| 98 | +creates Services for Postgres. In particular, we will want to use the ReadWrite |
| 99 | +service called `cluster-example-rw` for the migrations. |
| 100 | + |
| 101 | +We're going to use the `credentials` stanza of the AtlasSchema CRD referencing |
| 102 | +the password and the service. Following along the Atlas Operator Quickstart, we |
| 103 | +create a migration defining a table called `t1`. Save the following to a file |
| 104 | +named `atlas-schema.yaml`. |
| 105 | + |
| 106 | +``` yaml |
| 107 | +apiVersion: db.atlasgo.io/v1alpha1 |
| 108 | +kind: AtlasSchema |
| 109 | +metadata: |
| 110 | + name: atlasschema-pg |
| 111 | +spec: |
| 112 | + credentials: |
| 113 | + scheme: postgres |
| 114 | + host: cluster-example-rw.default |
| 115 | + user: app |
| 116 | + passwordFrom: |
| 117 | + secretKeyRef: |
| 118 | + key: password |
| 119 | + name: cluster-example-app |
| 120 | + database: app |
| 121 | + port: 5432 |
| 122 | + parameters: |
| 123 | + sslmode: disable |
| 124 | + schema: |
| 125 | + sql: | |
| 126 | + create table t1 ( |
| 127 | + id int |
| 128 | + ); |
| 129 | +``` |
| 130 | +
|
| 131 | +Then apply it: `kubectl apply -f atlas-schema.yaml`. |
| 132 | + |
| 133 | +You should soon be able to see the Schema has been applied: |
| 134 | + |
| 135 | +``` console |
| 136 | +> kubectl get atlasschemas.db.atlasgo.io |
| 137 | +NAME READY REASON |
| 138 | +atlasschema-pg True Applied |
| 139 | +``` |
| 140 | + |
| 141 | +To see the results, let's get a `psql` session open on one of our instances: |
| 142 | + |
| 143 | +``` console |
| 144 | +> kubectl exec -ti cluster-example-1 -- psql app |
| 145 | +Defaulted container "postgres" out of: postgres, bootstrap-controller (init) |
| 146 | +psql (17.4 (Debian 17.4-1.pgdg110+2)) |
| 147 | +Type "help" for help. |
| 148 | +
|
| 149 | +app=# \dt |
| 150 | + List of relations |
| 151 | + Schema | Name | Type | Owner |
| 152 | +--------+------+-------+------- |
| 153 | + public | t1 | table | app |
| 154 | +(1 row) |
| 155 | +
|
| 156 | +app=# \d t1 |
| 157 | + Table "public.t1" |
| 158 | + Column | Type | Collation | Nullable | Default |
| 159 | +--------+---------+-----------+----------+--------- |
| 160 | + id | integer | | | |
| 161 | +``` |
| 162 | + |
| 163 | +As suggested in the Atlas Quickstart, we can modify the schema in the |
| 164 | +`atlas-schema.yaml` file, and then re-apply: |
| 165 | +`kubectl apply -f atlas-schema.yaml`, and the Atlas operator will again |
| 166 | +reconcile the database to the desired state. |
| 167 | + |
| 168 | +This intro was just to whet your appetite. There is plenty more to learn with |
| 169 | +Atlas. |
0 commit comments