You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/postgresql/migrate/concepts-single-to-flexible.md
+16-15Lines changed: 16 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,7 @@ ms.author: shriramm
7
7
ms.reviewer: maghan
8
8
ms.date: 03/31/2023
9
9
ms.service: postgresql
10
+
ms.custom: references_regions
10
11
ms.topic: conceptual
11
12
---
12
13
@@ -28,7 +29,7 @@ In this article, we provide compelling reasons for single server customers to mi
28
29
29
30
-**[Cost Savings](../flexible-server/how-to-deploy-on-azure-free-account.md)** – Flexible server allows you to stop and start server on-demand to lower your TCO. Your compute tier billing is stopped immediately, which allows you to have significant cost savings during development, testing and for time-bound predictable production workloads.
30
31
31
-
-**[Support for new PG versions](../flexible-server/concepts-supported-versions.md)** - Flexible server currently supports PG version 11 and onwards till version 15. Newer community versions of PostgreSQL will be supported only in flexible server.
32
+
-**[Support for new PG versions](../flexible-server/concepts-supported-versions.md)** - Flexible server currently supports PG version 11 and onwards till version 15. Newer community versions of PostgreSQL are supported only in flexible server.
32
33
33
34
-**Minimized Latency** – You can collocate your flexible server in the same availability zone as the application server that results in a minimal latency. This option isn't available in Single server.
34
35
@@ -69,13 +70,13 @@ The following table lists the different tools available for performing the migra
69
70
| Tool | Mode | Pros | Cons |
70
71
| :--- | :--- | :--- | :--- |
71
72
| Single to Flex Migration tool (**Recommended**) | Offline | - Managed migration service.<br />- No complex setup/pre-requisites required<br />- Simple to use portal-based migration experience<br />- Fast offline migration tool<br />- No limitations in terms of size of databases it can handle. | Downtime to applications. |
72
-
| pg_dump and pg_restore | Offline | - Tried and tested tool that has been in use for long time<br />- Suited for databases of size less than 10 GB<br />| - Need prior knowledge of setting up and using this tool<br />- Slow when compared to other tools<br />Significant downtime to your application. |
73
+
| pg_dump and pg_restore | Offline | - Tried and tested tool that is in use for a long time<br />- Suited for databases of size less than 10 GB<br />| - Need prior knowledge of setting up and using this tool<br />- Slow when compared to other tools<br />Significant downtime to your application. |
73
74
| Azure DMS | Online | - Minimal downtime to your application<br />- Free of cost | - Complex setup<br />- High chances of migration failures<br />- Can't handle database of sizes > 1 TB<br />- Can't handle write-intensive workload |
74
75
75
76
The next section of the document gives an overview of the Single to Flex Migration tool, its implementation, limitations, and the experience that makes it the recommended tool to perform migrations from single to flexible server.
76
77
77
78
> [!NOTE]
78
-
> The Single to Flex Migration tool is available in all Azure regions and currently supports only **Offline** migrations. Support for online migrations will be introduced later in the tool.
79
+
> The Single to Flex Migration tool is available in all Azure regions and currently supports **Offline** migrations. Support for **Online** migrations is currently available in select regions - India Central, India South, Australia Southeast and South East Asia.
79
80
80
81
## Single to Flexible Migration tool - Overview
81
82
@@ -102,7 +103,7 @@ The following table shows the time for performing offline migrations for databas
102
103
> In order to perform faster migrations, pick a higher SKU for your flexible server. You can always change the SKU to match the application needs post migration.
103
104
104
105
## Pre-migration validations
105
-
We have noticed many migrations fail due to setup issues on source and target server. Most of the issues can be categorized into the following buckets:
106
+
We noticed many migrations fail due to setup issues on source and target server. Most of the issues can be categorized into the following buckets:
106
107
* Issues related to authentication/permissions for the migration user on source and target server.
107
108
*[Prerequisites](#migration-prerequisites) not being taken care of, before running the migration.
108
109
* Unsupported features/configurations between the source and target.
@@ -123,7 +124,7 @@ The result of the Validate option can be
123
124
124
125
Plan your migrations better by performing pre-migration validations in advance to know the potential issues you might encounter while performing migrations.
125
126
126
-
***Migrate** - Use this option to kickstart the migration without going through validation process. It's recommended to perform validation before triggering a migration to increase the chances for a successful migration. Once validation is done, you can use this option to start the migration process.
127
+
***Migrate** - Use this option to kickstart the migration without going through validation process. It's recommended to perform validation before triggering a migration to increase the chances for a successful migration. Once validation is done, you can use this option to start the migration process.
127
128
128
129
***Validate and Migrate** - In this option, validations are performed and then migration gets triggered if all checks are in **succeeded** or **warning** state. Validation failures don't start the migration between source and target servers.
129
130
@@ -156,7 +157,7 @@ Along with data migration, the tool automatically provides the following built-i
156
157
> [!NOTE]
157
158
> The following limitations are applicable only for flexible servers on which the migration of users/roles functionality is enabled.
158
159
159
-
- Azure Active Directory users present on your source server won't be migrated to target server. To mitigate this limitation, manually create all Azure Active Directory users on your target server using this [link](../flexible-server/how-to-manage-azure-ad-users.md) before triggering a migration. If Azure Active Directory users aren't created on target server, migration fails with appropriate error message.
160
+
- Azure Active Directory users present on your source server are not migrated to the target server. To mitigate this limitation, manually create all Azure Active Directory users on your target server using this [link](../flexible-server/how-to-manage-azure-ad-users.md) before triggering a migration. If Azure Active Directory users aren't created on target server, migration fails with appropriate error message.
160
161
- If the target flexible server uses SCRAM-SHA-256 password encryption method, connection to flexible server using the users/roles on single server fails since the passwords are encrypted using md5 algorithm. To mitigate this limitation, choose the option **MD5** for **password_encryption** server parameter on your flexible server.
161
162
## Experience
162
163
@@ -204,9 +205,9 @@ The following table summarizes the list of networking scenarios supported by the
204
205
- If your single server is public access (case #1 and case #2 in the above table), there's nothing needed from your end. The single to flex migration tool automatically establishes connection between single and flexible server and the migration goes through.
205
206
- If your single server is in private access, then the only supported scenario is when your Flexible server is inside a VNet. If your flexible server is deployed in the same VNet as the private end point of your Single server, connections between single server and flexible server should automatically work provided there's no network security group(NSGs) blocking the connectivity between subnets. If flexible server is deployed in another VNet, [peering should be established between the VNets](../../virtual-network/tutorial-connect-virtual-networks-portal.md) for the connection to work between Single and Flexible server.
206
207
207
-
##### Allow list required extensions
208
+
##### Allowlist required extensions
208
209
209
-
The migration tool automatically allows lists all extensions used by your single server databases on your flexible server except for the ones whose libraries need to be loaded at the server start.
210
+
The migration tool automatically allowlists all extensions used by your single server databases on your flexible server except for the ones whose libraries need to be loaded at the server start.
210
211
211
212
Use the following select command to list all the extensions used on your Single server databases.
212
213
@@ -239,7 +240,7 @@ Use the **Save and Restart** option and wait for the flexible server to restart.
239
240
> [!NOTE]
240
241
> If TIMESCALEDB, POSTGIS_TOPOLOGY, POSTGIS_TIGER_GEOCODER, POSTGRES_FDW or PG_PARTMAN extensions are used in your single server, please raise a support request since the migration tool does not handle these extensions.
241
242
242
-
##### Create AAD users on target server
243
+
##### Create Azure Active Directory users on target server
243
244
> [!NOTE]
244
245
> This pre-requisite is applicable only for flexible servers on which the migration of users/roles functionality is enabled.
245
246
@@ -269,7 +270,7 @@ The next phase of planning involves downtime incurred by applications for perfor
269
270
270
271
In most cases, the non-prod servers (dev, UAT, test, staging) are migrated using offline migrations. Since these servers have less data than the production servers, the migration completes fast. For migration of production server, you need to know the time it would take to complete the migration to plan for it in advance.
271
272
272
-
The time taken for an offline migration to complete is dependent on several factors that includes the number of databases, size of databases, number of tables inside each database, number of indexes, and the distribution of data across tables. It also depends on the SKU of the source and target server, and the IOPS available on the source and target server. Given the many factors that can affect the migration time, it's hard to estimate the total time for the offline migration to complete. The best approach would be to try it on a server restored from the primary server.
273
+
The time taken for an offline migration to complete depends on several factors. It includes the number of databases, size of databases, number of tables inside each database, number of indexes, and the distribution of data across tables. It also depends on the SKU of the source and target server, and the IOPS available on the source and target server. Given the many factors that can affect the migration time, it's hard to estimate the total time for the offline migration to complete. The best approach would be to try it on a server restored from the primary server.
273
274
274
275
For calculating the total downtime to perform offline migration of production server, the following phases are considered.
275
276
@@ -287,7 +288,7 @@ For calculating the total downtime to perform offline migration of production se
287
288
288
289
-**Migration of server settings** - The server parameters, firewall rules (if applicable), tags, alerts need to be manually copied from single server to flexible server.
289
290
290
-
-**Changing connection strings** - Post successful validation, application should change their connection strings to point to flexible server. This activity is coordinated with the application team to make changes to all the references of connection strings pointing to single server. In the flexible server the user parameter in the connection string no longer needs to be in the **username@servername** format. You should just use the **user=username** format for this parameter in the connection string
291
+
-**Changing connection strings** - Post successful validation, application should change their connection strings to point to flexible server. This activity is coordinated with the application team to make changes to all the references of connection strings pointing to single server. In the flexible server, the user parameter in the connection string no longer needs to be in the **username@servername** format. You should just use the **user=username** format for this parameter in the connection string
@@ -313,7 +314,7 @@ Trigger the migration of your production databases using the **Migrate** or **Va
313
314
314
315
In general, a powerful SKU is recommended for the target as the migration tool runs out of a container on the Flexible server. A powerful SKU enables a greater number of tables to be migrated in parallel. You can scale the SKU back to your preferred configuration after the migration. This section contains steps to improve the migration speed in case the data distribution among the tables is skewed and/or a more powerful SKU doesn't have a significant impact on the migration speed.
315
316
316
-
If the data distribution on the source is highly skewed, with most of the data present in one table, the allocated compute for migration isn't fully utilized and it creates a bottleneck. So, we split large tables into smaller chunks, which are then migrated in parallel. This is applicable to tables that have more than 10000000 (10m) tuples. Splitting the table into smaller chunks is possible if one of the following conditions is satisfied.
317
+
If the data distribution on the source is highly skewed, with most of the data present in one table, the allocated compute for migration isn't fully utilized and it creates a bottleneck. So, we split large tables into smaller chunks, which are then migrated in parallel. This feature is applicable to tables that have more than 10000000 (10 m) tuples. Splitting the table into smaller chunks is possible if one of the following conditions is satisfied.
317
318
318
319
1. The table must have a column with a simple (not composite) primary key or unique index of type int or big int.
319
320
@@ -337,20 +338,20 @@ If any of the above conditions are satisfied, the table is migrated in multiple
337
338
##### How it works
338
339
339
340
- The migration tool looks up the maximum and minimum integer value of the Primary key/Unique index of that table that must be split up and migrated in parallel.
340
-
- If the difference between the minimum and maximum value is more than 10000000 (10m), then the table is split into multiple parts and each part is migrated separately, in parallel.
341
+
- If the difference between the minimum and maximum value is more than 10000000 (10 m), then the table is split into multiple parts and each part is migrated separately, in parallel.
341
342
342
343
In summary, the Single to Flexible migration tool migrates a table in parallel threads and reduce the migration time if:
343
344
344
345
1. The table has a column with a simple primary key or unique index of type int or big int.
345
-
2. The table has at least 10000000 (10m) rows so that the difference between the minimum and maximum value of the primary key is more than 10000000 (10m).
346
+
2. The table has at least 10000000 (10 m) rows so that the difference between the minimum and maximum value of the primary key is more than 10000000 (10 m).
346
347
3. The SKU used has idle cores, which can be used for migrating the table in parallel.
347
348
348
349
### Post migration
349
350
350
351
- Once the migration is complete, verify the data on your flexible server and make sure it's an exact copy of the single server.
351
352
- Post verification, enable HA option as needed on your flexible server.
352
353
- Change the SKU of the flexible server to match the application needs. This change needs a database server restart.
353
-
- If you've changed any server parameters from their default values in single server, copy those server parameter values in flexible server.
354
+
- If you change any server parameters from their default values in single server, copy those server parameter values in flexible server.
354
355
- Copy other server settings like tags, alerts, firewall rules (if applicable) from single server to flexible server.
355
356
- Make changes to your application to point the connection strings to flexible server.
356
357
- Monitor the database performance closely to see if it requires performance tuning.
0 commit comments