Skip to content

Commit 636ea9c

Browse files
HJToland3HJToland3
committed
postgreSQL updates
1 parent 0375f63 commit 636ea9c

4 files changed

+45
-129
lines changed

‎articles/dms/known-issues-azure-postgresql-online.md

Lines changed: 19 additions & 111 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: "Known issues: Online migrations from PostgreSQL to Azure Database for PostgreSQL"
33
titleSuffix: Azure Database Migration Service
4-
description: Learn about known issues and migration limitations with online migrations from PostgreSQL to Azure Database for PostgreSQL-Single server using the Azure Database Migration Service.
4+
description: Learn about known issues and migration limitations with online migrations from PostgreSQL to Azure Database for PostgreSQ using the Azure Database Migration Service.
55
services: database-migration
66
author: HJToland3
77
ms.author: jtoland
@@ -11,34 +11,32 @@ ms.service: dms
1111
ms.workload: data-services
1212
ms.custom: [seo-lt-2019, seo-dt-2019]
1313
ms.topic: article
14-
ms.date: 10/27/2019
14+
ms.date: 02/13/2020
1515
---
1616

17-
# Known issues/migration limitations with online migrations from PostgreSQL to Azure DB for PostgreSQL-Single server
17+
# Known issues/migration limitations with online migrations from PostgreSQL to Azure DB for PostgreSQL
1818

19-
Known issues and limitations associated with online migrations from PostgreSQL to Azure Database for PostgreSQL-Single server are described in the following sections.
19+
Known issues and limitations associated with online migrations from PostgreSQL to Azure Database for PostgreSQL are described in the following sections.
2020

2121
## Online migration configuration
2222

23-
- The source PostgreSQL server must be running version 9.5.11, 9.6.7, or 10.3 or later. For more information, see the article [Supported PostgreSQL Database Versions](../postgresql/concepts-supported-versions.md).
24-
- Only same version migrations are supported. For example, migrating PostgreSQL 9.5.11 to Azure Database for PostgreSQL 9.6.7 is not supported.
25-
26-
> [!NOTE]
27-
> For PostgreSQL version 10, currently DMS only supports migration of version 10.3 to Azure Database for PostgreSQL. We are planning to support newer versions of PostgreSQL very soon.
28-
23+
- The source PostgreSQL server must be running version 9.4, 9.5, 9.6, 10, or 11. For more information, see the article [Supported PostgreSQL Database Versions](../postgresql/concepts-supported-versions.md).
24+
- Only migrations to the same or a higher version are supported. For example, migrating PostgreSQL 9.5 to Azure Database for PostgreSQL 9.6 or 10 is supported, but migrating from PostgreSQL 11 to PostgreSQL 9.6 isn't supported.
2925
- To enable logical replication in the **source PostgreSQL postgresql.conf** file, set the following parameters:
3026
- **wal_level** = logical
31-
- **max_replication_slots** = [max number of databases for migration]; if you want to migrate four databases, set the value to 4
27+
- **max_replication_slots** = [at least max number of databases for migration]; if you want to migrate four databases, set the value to at least 4.
3228
- **max_wal_senders** = [number of databases running concurrently]; the recommended value is 10
3329
- Add DMS agent IP to the source PostgreSQL pg_hba.conf
34-
1. Make a note of the DMS IP address after you finish provisioning an instance of DMS.
30+
1. Make a note of the DMS IP address after you finish provisioning an instance of Azure Database Migration Service.
3531
2. Add the IP address to the pg_hba.conf file as shown:
3632

37-
host all 172.16.136.18/10 md5
38-
host replication postgres 172.16.136.18/10 md5
33+
```
34+
host all 172.16.136.18/10 md5
35+
host replication postgres 172.16.136.18/10 md5
36+
```
3937
40-
- The user must have the super user permission on the server hosting the source database
41-
- Aside from having ENUM in the source database schema, the source and target database schemas must match.
38+
- The user must have the REPLICATION role on the server hosting the source database.
39+
- The source and target database schemas must match.
4240
- The schema in the target Azure Database for PostgreSQL-Single server must not have foreign keys. Use the following query to drop foreign keys:
4341
4442
```
@@ -78,81 +76,10 @@ Known issues and limitations associated with online migrations from PostgreSQL t
7876
7977
## Datatype limitations
8078
81-
- **Limitation**: If there's an ENUM datatype in the source PostgreSQL database, the migration will fail during continuous sync.
82-
83-
**Workaround**: Modify ENUM datatype to character varying in Azure Database for PostgreSQL.
84-
85-
- **Limitation**: If there's no primary key on tables, continuous sync will fail.
79+
- **Limitation**: If there's no primary key on tables, changes may not be synced to the target database.
8680
8781
**Workaround**: Temporarily set a primary key for the table for migration to continue. You can remove the primary key after data migration is complete.
8882
89-
- **Limitation**: JSONB datatype not supported for migration.
90-
91-
## LOB limitations
92-
93-
Large Object (LOB) columns are columns that can grow large. For PostgreSQL, examples of LOB data types include XML, JSON, IMAGE, TEXT, etc.
94-
95-
- **Limitation**: If LOB data types are used as primary keys, migration will fail.
96-
97-
**Workaround**: Replace primary key with other datatypes or columns that aren't LOB.
98-
99-
- **Limitation**: If the length of Large Object (LOB) column is bigger than 32 KB, data might be truncated at the target. You can check the length of LOB column using this query:
100-
101-
```
102-
SELECT max(length(cast(body as text))) as body FROM customer_mail
103-
```
104-
105-
**Workaround**: If you have LOB object that is bigger than 32 KB, contact engineering team at [Ask Azure Database Migrations](mailto:[email protected]).
106-
107-
- **Limitation**: If there are LOB columns in the table, and there's no primary key set for the table, data might not be migrated for this table.
108-
109-
**Workaround**: Temporarily set a primary key for the table for migration to proceed. You can remove the primary key after data migration is complete.
110-
111-
## PostgreSQL10 workaround
112-
113-
PostgreSQL 10.x makes various changes to pg_xlog folder names and hence causing migration not running as expected. If you're migrating from PostgreSQL 10.x to Azure Database for PostgreSQL 10.3, execute the following script on the source PostgreSQL database to create wrapper function around pg_xlog functions.
114-
115-
```
116-
BEGIN;
117-
CREATE SCHEMA IF NOT EXISTS fnRenames;
118-
CREATE OR REPLACE FUNCTION fnRenames.pg_switch_xlog() RETURNS pg_lsn AS $$
119-
SELECT pg_switch_wal(); $$ LANGUAGE SQL;
120-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlog_replay_pause() RETURNS VOID AS $$
121-
SELECT pg_wal_replay_pause(); $$ LANGUAGE SQL;
122-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlog_replay_resume() RETURNS VOID AS $$
123-
SELECT pg_wal_replay_resume(); $$ LANGUAGE SQL;
124-
CREATE OR REPLACE FUNCTION fnRenames.pg_current_xlog_location() RETURNS pg_lsn AS $$
125-
SELECT pg_current_wal_lsn(); $$ LANGUAGE SQL;
126-
CREATE OR REPLACE FUNCTION fnRenames.pg_is_xlog_replay_paused() RETURNS boolean AS $$
127-
SELECT pg_is_wal_replay_paused(); $$ LANGUAGE SQL;
128-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlogfile_name(lsn pg_lsn) RETURNS TEXT AS $$
129-
SELECT pg_walfile_name(lsn); $$ LANGUAGE SQL;
130-
CREATE OR REPLACE FUNCTION fnRenames.pg_last_xlog_replay_location() RETURNS pg_lsn AS $$
131-
SELECT pg_last_wal_replay_lsn(); $$ LANGUAGE SQL;
132-
CREATE OR REPLACE FUNCTION fnRenames.pg_last_xlog_receive_location() RETURNS pg_lsn AS $$
133-
SELECT pg_last_wal_receive_lsn(); $$ LANGUAGE SQL;
134-
CREATE OR REPLACE FUNCTION fnRenames.pg_current_xlog_flush_location() RETURNS pg_lsn AS $$
135-
SELECT pg_current_wal_flush_lsn(); $$ LANGUAGE SQL;
136-
CREATE OR REPLACE FUNCTION fnRenames.pg_current_xlog_insert_location() RETURNS pg_lsn AS $$
137-
SELECT pg_current_wal_insert_lsn(); $$ LANGUAGE SQL;
138-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlog_location_diff(lsn1 pg_lsn, lsn2 pg_lsn) RETURNS NUMERIC AS $$
139-
SELECT pg_wal_lsn_diff(lsn1, lsn2); $$ LANGUAGE SQL;
140-
CREATE OR REPLACE FUNCTION fnRenames.pg_xlogfile_name_offset(lsn pg_lsn, OUT TEXT, OUT INTEGER) AS $$
141-
SELECT pg_walfile_name_offset(lsn); $$ LANGUAGE SQL;
142-
CREATE OR REPLACE FUNCTION fnRenames.pg_create_logical_replication_slot(slot_name name, plugin name,
143-
temporary BOOLEAN DEFAULT FALSE, OUT slot_name name, OUT xlog_position pg_lsn) RETURNS RECORD AS $$
144-
SELECT slot_name::NAME, lsn::pg_lsn FROM pg_catalog.pg_create_logical_replication_slot(slot_name, plugin,
145-
temporary); $$ LANGUAGE SQL;
146-
ALTER USER PG_User SET search_path = fnRenames, pg_catalog, "$user", public;
147-
148-
-- DROP SCHEMA fnRenames CASCADE;
149-
-- ALTER USER PG_User SET search_path TO DEFAULT;
150-
COMMIT;
151-
```
152-
153-
> [!NOTE]
154-
> In the preceding script, "PG_User" refers to the username being used to connect to the migration source.
155-
15683
## Limitations when migrating online from AWS RDS PostgreSQL
15784
15885
When you try to perform an online migration from AWS RDS PostgreSQL to Azure Database for PostgreSQL, you may encounter the following errors.
@@ -169,38 +96,19 @@ When you try to perform an online migration from AWS RDS PostgreSQL to Azure Dat
16996
17097
- **Error:** The source database {database} is empty.
17198
172-
**Limitation**: This error occurs when the source database is empty. This is most likely because you have selected the wrong database as source.
99+
**Limitation**: This error occurs when the source database is empty. It is most likely because you have selected the wrong database as source.
173100
**Workaround**: Double-check the source database you selected for migration, and then try again.
174101
175102
- **Error:** The target database {database} is empty. Please migrate the schema.
176103
177-
**Limitation**: This error occurs when there is no schema on the target database. Make sure schema on the target matches schema on the source.
104+
**Limitation**: This error occurs when there's no schema on the target database. Make sure schema on the target matches schema on the source.
178105
**Workaround**: Ensure that the schema on the target matches schema on the source. For detail on migrating schema, refer to the [Azure PostgreSQL online migration documentation](https://docs.microsoft.com/azure/dms/tutorial-postgresql-azure-postgresql-online#migrate-the-sample-schema).
179106
180107
## Other limitations
181108
182109
- The database name can't include a semi-colon (;).
183-
- Password string that has opening and closing curly brackets { } isn't supported. This limitation applies to both connecting to source PostgreSQL and target Azure Database for PostgreSQL.
184110
- A captured table must have a Primary Key. If a table doesn't have a primary key, the result of DELETE and UPDATE record operations will be unpredictable.
185111
- Updating a Primary Key segment is ignored. In such cases, applying such an update will be identified by the target as an update that didn't update any rows and will result in a record written to the exceptions table.
186112
- Migration of multiple tables with the same name but a different case (e.g. table1, TABLE1, and Table1) may cause unpredictable behavior and is therefore not supported.
187-
- Change processing of [CREATE | ALTER | DROP] table DDLs are supported unless they are held in an inner function/procedure body block or in other nested constructs. For example, the following change will not be captured:
188-
189-
```
190-
CREATE OR REPLACE FUNCTION pg.create_distributors1() RETURNS void
191-
LANGUAGE plpgsql
192-
AS $$
193-
BEGIN
194-
create table pg.distributors1(did serial PRIMARY KEY,name varchar(40)
195-
NOT NULL);
196-
END;
197-
$$;
198-
```
199-
200-
- Change processing (continuous sync) of TRUNCATE operations isn't supported. Migration of partitioned tables is not supported. When a partitioned table is detected, the following things occur:
201-
202-
- The database will report a list of parent and child tables.
203-
- The table will be created on the target as a regular table with the same properties as the selected tables.
204-
- If the parent table in the source database has the same Primary Key value as its child tables, a “duplicate key” error will be generated.
205-
206-
- In DMS, the limit of databases to migrate in one single migration activity is four.
113+
- Change processing of [CREATE | ALTER | DROP | TRUNCATE] table DDLs isn't supported.
114+
- In Azure Database Migration Service, a single migration activity can only accommodate up to four databases.

0 commit comments

Comments
 (0)