You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: "Known issues: Online migrations from PostgreSQL to Azure Database for PostgreSQL"
3
3
titleSuffix: Azure Database Migration Service
4
-
description: Learn about known issues and migration limitations with online migrations from PostgreSQL to Azure Database for PostgreSQL-Single server using the Azure Database Migration Service.
4
+
description: Learn about known issues and migration limitations with online migrations from PostgreSQL to Azure Database for PostgreSQ using the Azure Database Migration Service.
5
5
services: database-migration
6
6
author: HJToland3
7
7
ms.author: jtoland
@@ -11,34 +11,32 @@ ms.service: dms
11
11
ms.workload: data-services
12
12
ms.custom: [seo-lt-2019, seo-dt-2019]
13
13
ms.topic: article
14
-
ms.date: 10/27/2019
14
+
ms.date: 02/13/2020
15
15
---
16
16
17
-
# Known issues/migration limitations with online migrations from PostgreSQL to Azure DB for PostgreSQL-Single server
17
+
# Known issues/migration limitations with online migrations from PostgreSQL to Azure DB for PostgreSQL
18
18
19
-
Known issues and limitations associated with online migrations from PostgreSQL to Azure Database for PostgreSQL-Single server are described in the following sections.
19
+
Known issues and limitations associated with online migrations from PostgreSQL to Azure Database for PostgreSQL are described in the following sections.
20
20
21
21
## Online migration configuration
22
22
23
-
- The source PostgreSQL server must be running version 9.5.11, 9.6.7, or 10.3 or later. For more information, see the article [Supported PostgreSQL Database Versions](../postgresql/concepts-supported-versions.md).
24
-
- Only same version migrations are supported. For example, migrating PostgreSQL 9.5.11 to Azure Database for PostgreSQL 9.6.7 is not supported.
25
-
26
-
> [!NOTE]
27
-
> For PostgreSQL version 10, currently DMS only supports migration of version 10.3 to Azure Database for PostgreSQL. We are planning to support newer versions of PostgreSQL very soon.
28
-
23
+
- The source PostgreSQL server must be running version 9.4, 9.5, 9.6, 10, or 11. For more information, see the article [Supported PostgreSQL Database Versions](../postgresql/concepts-supported-versions.md).
24
+
- Only migrations to the same or a higher version are supported. For example, migrating PostgreSQL 9.5 to Azure Database for PostgreSQL 9.6 or 10 is supported, but migrating from PostgreSQL 11 to PostgreSQL 9.6 isn't supported.
29
25
- To enable logical replication in the **source PostgreSQL postgresql.conf** file, set the following parameters:
30
26
-**wal_level** = logical
31
-
-**max_replication_slots** = [max number of databases for migration]; if you want to migrate four databases, set the value to 4
27
+
-**max_replication_slots** = [at least max number of databases for migration]; if you want to migrate four databases, set the value to at least 4.
32
28
-**max_wal_senders** = [number of databases running concurrently]; the recommended value is 10
33
29
- Add DMS agent IP to the source PostgreSQL pg_hba.conf
34
-
1. Make a note of the DMS IP address after you finish provisioning an instance of DMS.
30
+
1. Make a note of the DMS IP address after you finish provisioning an instance of Azure Database Migration Service.
35
31
2. Add the IP address to the pg_hba.conf file as shown:
36
32
37
-
host all 172.16.136.18/10 md5
38
-
host replication postgres 172.16.136.18/10 md5
33
+
```
34
+
host all 172.16.136.18/10 md5
35
+
host replication postgres 172.16.136.18/10 md5
36
+
```
39
37
40
-
- The user must have the super user permission on the server hosting the source database
41
-
-Aside from having ENUM in the source database schema, the source and target database schemas must match.
38
+
- The user must have the REPLICATION role on the server hosting the source database.
39
+
- The source and target database schemas must match.
42
40
- The schema in the target Azure Database for PostgreSQL-Single server must not have foreign keys. Use the following query to drop foreign keys:
43
41
44
42
```
@@ -78,80 +76,9 @@ Known issues and limitations associated with online migrations from PostgreSQL t
78
76
79
77
## Datatype limitations
80
78
81
-
- **Limitation**: If there's an ENUM datatype in the source PostgreSQL database, the migration will fail during continuous sync.
82
-
83
-
**Workaround**: Modify ENUM datatype to character varying in Azure Database for PostgreSQL.
84
-
85
-
- **Limitation**: If there's no primary key on tables, continuous sync will fail.
86
-
87
-
**Workaround**: Temporarily set a primary key for the table for migration to continue. You can remove the primary key after data migration is complete.
88
-
89
-
- **Limitation**: JSONB datatype not supported for migration.
90
-
91
-
## LOB limitations
92
-
93
-
Large Object (LOB) columns are columns that can grow large. For PostgreSQL, examples of LOB data types include XML, JSON, IMAGE, TEXT, etc.
94
-
95
-
- **Limitation**: If LOB data types are used as primary keys, migration will fail.
96
-
97
-
**Workaround**: Replace primary key with other datatypes or columns that aren't LOB.
98
-
99
-
- **Limitation**: If the length of Large Object (LOB) column is bigger than 32 KB, data might be truncated at the target. You can check the length of LOB column using this query:
79
+
**Limitation**: If there's no primary key on tables, changes may not be synced to the target database.
100
80
101
-
```
102
-
SELECT max(length(cast(body as text))) as body FROM customer_mail
103
-
```
104
-
105
-
**Workaround**: If you have LOB object that is bigger than 32 KB, contact engineering team at [Ask Azure Database Migrations](mailto:[email protected]).
106
-
107
-
- **Limitation**: If there are LOB columns in the table, and there's no primary key set for the table, data might not be migrated for this table.
108
-
109
-
**Workaround**: Temporarily set a primary key for the table for migration to proceed. You can remove the primary key after data migration is complete.
110
-
111
-
## PostgreSQL10 workaround
112
-
113
-
PostgreSQL 10.x makes various changes to pg_xlog folder names and hence causing migration not running as expected. If you're migrating from PostgreSQL 10.x to Azure Database for PostgreSQL 10.3, execute the following script on the source PostgreSQL database to create wrapper function around pg_xlog functions.
114
-
115
-
```
116
-
BEGIN;
117
-
CREATE SCHEMA IF NOT EXISTS fnRenames;
118
-
CREATE OR REPLACE FUNCTION fnRenames.pg_switch_xlog() RETURNS pg_lsn AS $$
119
-
SELECT pg_switch_wal(); $$ LANGUAGE SQL;
120
-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlog_replay_pause() RETURNS VOID AS $$
121
-
SELECT pg_wal_replay_pause(); $$ LANGUAGE SQL;
122
-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlog_replay_resume() RETURNS VOID AS $$
123
-
SELECT pg_wal_replay_resume(); $$ LANGUAGE SQL;
124
-
CREATE OR REPLACE FUNCTION fnRenames.pg_current_xlog_location() RETURNS pg_lsn AS $$
125
-
SELECT pg_current_wal_lsn(); $$ LANGUAGE SQL;
126
-
CREATE OR REPLACE FUNCTION fnRenames.pg_is_xlog_replay_paused() RETURNS boolean AS $$
127
-
SELECT pg_is_wal_replay_paused(); $$ LANGUAGE SQL;
128
-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlogfile_name(lsn pg_lsn) RETURNS TEXT AS $$
129
-
SELECT pg_walfile_name(lsn); $$ LANGUAGE SQL;
130
-
CREATE OR REPLACE FUNCTION fnRenames.pg_last_xlog_replay_location() RETURNS pg_lsn AS $$
131
-
SELECT pg_last_wal_replay_lsn(); $$ LANGUAGE SQL;
132
-
CREATE OR REPLACE FUNCTION fnRenames.pg_last_xlog_receive_location() RETURNS pg_lsn AS $$
133
-
SELECT pg_last_wal_receive_lsn(); $$ LANGUAGE SQL;
134
-
CREATE OR REPLACE FUNCTION fnRenames.pg_current_xlog_flush_location() RETURNS pg_lsn AS $$
135
-
SELECT pg_current_wal_flush_lsn(); $$ LANGUAGE SQL;
136
-
CREATE OR REPLACE FUNCTION fnRenames.pg_current_xlog_insert_location() RETURNS pg_lsn AS $$
137
-
SELECT pg_current_wal_insert_lsn(); $$ LANGUAGE SQL;
138
-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlog_location_diff(lsn1 pg_lsn, lsn2 pg_lsn) RETURNS NUMERIC AS $$
139
-
SELECT pg_wal_lsn_diff(lsn1, lsn2); $$ LANGUAGE SQL;
140
-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlogfile_name_offset(lsn pg_lsn, OUT TEXT, OUT INTEGER) AS $$
141
-
SELECT pg_walfile_name_offset(lsn); $$ LANGUAGE SQL;
142
-
CREATE OR REPLACE FUNCTION fnRenames.pg_create_logical_replication_slot(slot_name name, plugin name,
143
-
temporary BOOLEAN DEFAULT FALSE, OUT slot_name name, OUT xlog_position pg_lsn) RETURNS RECORD AS $$
144
-
SELECT slot_name::NAME, lsn::pg_lsn FROM pg_catalog.pg_create_logical_replication_slot(slot_name, plugin,
145
-
temporary); $$ LANGUAGE SQL;
146
-
ALTER USER PG_User SET search_path = fnRenames, pg_catalog, "$user", public;
147
-
148
-
-- DROP SCHEMA fnRenames CASCADE;
149
-
-- ALTER USER PG_User SET search_path TO DEFAULT;
150
-
COMMIT;
151
-
```
152
-
153
-
> [!NOTE]
154
-
> In the preceding script, "PG_User" refers to the username being used to connect to the migration source.
81
+
**Workaround**: Temporarily set a primary key for the table for migration to continue. You can remove the primary key after data migration is complete.
155
82
156
83
## Limitations when migrating online from AWS RDS PostgreSQL
157
84
@@ -165,42 +92,25 @@ When you try to perform an online migration from AWS RDS PostgreSQL to Azure Dat
165
92
- **Error**: Target database '{database}' has '{number of tables}' tables where as source database '{database}' has '{number of tables}' tables. The number of tables on source and target databases should match.
166
93
167
94
**Limitation**: This error occurs when the number of tables is different between the source and target databases.
95
+
168
96
**Workaround**: Ensure that the schema on the target matches schema on the source. For detail on migrating schema, refer to the [Azure PostgreSQL online migration documentation](https://docs.microsoft.com/azure/dms/tutorial-postgresql-azure-postgresql-online#migrate-the-sample-schema).
169
97
170
98
- **Error:** The source database {database} is empty.
171
99
172
-
**Limitation**: This error occurs when the source database is empty. This is most likely because you have selected the wrong database as source.
100
+
**Limitation**: This error occurs when the source database is empty. It is most likely because you have selected the wrong database as source.
101
+
173
102
**Workaround**: Double-check the source database you selected for migration, and then try again.
174
103
175
104
- **Error:** The target database {database} is empty. Please migrate the schema.
176
105
177
-
**Limitation**: This error occurs when there is no schema on the target database. Make sure schema on the target matches schema on the source.
106
+
**Limitation**: This error occurs when there's no schema on the target database. Make sure schema on the target matches schema on the source.
178
107
**Workaround**: Ensure that the schema on the target matches schema on the source. For detail on migrating schema, refer to the [Azure PostgreSQL online migration documentation](https://docs.microsoft.com/azure/dms/tutorial-postgresql-azure-postgresql-online#migrate-the-sample-schema).
179
108
180
109
## Other limitations
181
110
182
111
- The database name can't include a semi-colon (;).
183
-
- Password string that has opening and closing curly brackets { } isn't supported. This limitation applies to both connecting to source PostgreSQL and target Azure Database for PostgreSQL.
184
112
- A captured table must have a Primary Key. If a table doesn't have a primary key, the result of DELETE and UPDATE record operations will be unpredictable.
185
113
- Updating a Primary Key segment is ignored. In such cases, applying such an update will be identified by the target as an update that didn't update any rows and will result in a record written to the exceptions table.
186
114
- Migration of multiple tables with the same name but a different case (e.g. table1, TABLE1, and Table1) may cause unpredictable behavior and is therefore not supported.
187
-
- Change processing of [CREATE | ALTER | DROP] table DDLs are supported unless they are held in an inner function/procedure body block or in other nested constructs. For example, the following change will not be captured:
188
-
189
-
```
190
-
CREATE OR REPLACE FUNCTION pg.create_distributors1() RETURNS void
191
-
LANGUAGE plpgsql
192
-
AS $$
193
-
BEGIN
194
-
create table pg.distributors1(did serial PRIMARY KEY,name varchar(40)
195
-
NOT NULL);
196
-
END;
197
-
$$;
198
-
```
199
-
200
-
- Change processing (continuous sync) of TRUNCATE operations isn't supported. Migration of partitioned tables is not supported. When a partitioned table is detected, the following things occur:
201
-
202
-
- The database will report a list of parent and child tables.
203
-
- The table will be created on the target as a regular table with the same properties as the selected tables.
204
-
- If the parent table in the source database has the same Primary Key value as its child tables, a “duplicate key” error will be generated.
205
-
206
-
- In DMS, the limit of databases to migrate in one single migration activity is four.
115
+
- Change processing of [CREATE | ALTER | DROP | TRUNCATE] table DDLs isn't supported.
116
+
- In Azure Database Migration Service, a single migration activity can only accommodate up to four databases.
0 commit comments