Skip to content

Commit 1bed850

Browse files
add schema migration
1 parent 86362ae commit 1bed850

File tree

2 files changed

+38
-1
lines changed

2 files changed

+38
-1
lines changed

docs/cloud/managed-postgres/migrations/peerdb.md

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ import createMirror from '@site/static/images/managed-postgres/peerdb/create-mir
1616
import tablePicker from '@site/static/images/managed-postgres/peerdb/table-picker.png';
1717
import initialLoad from '@site/static/images/managed-postgres/peerdb/initial-load.png';
1818
import mirrors from '@site/static/images/managed-postgres/peerdb/mirrors.png';
19+
import settings from '@site/static/images/managed-postgres/peerdb/settings.png';
1920

2021
# Migrate to Managed Postgres using PeerDB {#peerdb-migration}
2122
This guide provides step-by-step instructions on how to migrate your PostgreSQL database to ClickHouse Managed Postgres using PeerDB.
@@ -47,14 +48,50 @@ Similarly, create a peer for your ClickHouse Managed Postgres instance by provid
4748
Now, you should see both the source and target peers listed in the "Peers" section.
4849
<Image img={peers} alt="Peers List" size="md" border />
4950

51+
### Obtain source schema dump {#migration-peerdb-source-schema-dump}
52+
To mirror the setup of the source database in the target database, we need to obtain a schema dump of the source database. You can use `pg_dump` to create a schema-only dump of your source PostgreSQL database:
53+
```shell
54+
pg_dump -d 'postgresql://<user>:<password>@<host>:<port>/<database>' -s > source_schema.sql
55+
```
56+
57+
Before applying this to the target database, we need to remove UNIQUE constraints and indexes from the dump file so that PeerDB's ingestion to target tables is not blocked by these constraints. These can be removed using:
58+
```shell
59+
# Preview
60+
grep -n "CONSTRAINT.*UNIQUE" <dump_file_path>
61+
grep -n "CREATE UNIQUE INDEX" <dump_file_path>
62+
grep -n -E "(CONSTRAINT.*UNIQUE|CREATE UNIQUE INDEX)" <dump_file_path>
63+
64+
# Remove
65+
sed -i.bak -E '/CREATE UNIQUE INDEX/,/;/d; /(CONSTRAINT.*UNIQUE|ADD CONSTRAINT.*UNIQUE)/d' <dump_file_path>
66+
```
67+
68+
### Apply schema dump to target database {#migration-peerdb-apply-schema-dump}
69+
After cleaning up the schema dump file, you can apply it to your target ClickHouse Managed Postgres database by [connecting](../connection) via `psql` and running the schema dump file:
70+
```shell
71+
psql -h <target_host> -p <target_port> -U <target_username> -d <target_database> -f source_schema.sql
72+
```
73+
74+
Here on the target side, we do not want PeerDB ingestion to be blocked by foreign key constraints. For this, we can alter the target role (used above in the target peer) to have `session_replication_role` set to `replica`:
75+
```sql
76+
ALTER ROLE <target_role> SET session_replication_role = replica;
77+
```
78+
5079
## Create a mirror {#migration-peerdb-create-mirror}
5180
Next, we need to create a mirror to define the data migration process between the source and target peers. In PeerDB UI, navigate to the "Mirrors" section by clicking on "Mirrors" in the sidebar. To create a new mirror, click on the `+ New mirror` button.
5281
<Image img={createMirror} alt="Create Mirror" size="md" border />
5382
1. Give your mirror a name that describes the migration.
5483
2. Select the source and target peers you created earlier from the dropdown menus.
55-
3. You may choose to enable continuous replication if you want to keep the target database in sync with the source after the initial migration. Otherwise, under **Advanced settings**, you can enable **Initial copy only** to perform a one-time migration.
84+
3. Make sure that:
85+
- Soft delete is OFF.
86+
- Expand `Advanced settings`. Make sure that the **Postgres type system is enabled** and **PeerDB columns are disabled**.
87+
<Image img={settings} alt="Mirror Settings" size="md" border />
5688
4. Select the tables you want to migrate. You can choose specific tables or select all tables from the source database.
5789
<Image img={tablePicker} alt="Table Picker" size="md" border />
90+
91+
:::info Selecting tables
92+
Make sure the destination table names are the same as the source table names in the target database, as we have migrated the schema as is in the earlier step.
93+
:::
94+
5895
5. Once you have configured the mirror settings, click on the `Create mirror` button.
5996

6097
You should see your newly created mirror in the "Mirrors" section.
961 KB
Loading

0 commit comments

Comments
 (0)