You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cosmos-db/migration-choices.md
+8-12Lines changed: 8 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,7 @@ ms.date: 04/02/2022
15
15
You can load data from various data sources to Azure Cosmos DB. Since Azure Cosmos DB supports multiple APIs, the targets can be any of the existing APIs. The following are some scenarios where you migrate data to Azure Cosmos DB:
16
16
17
17
* Move data from one Azure Cosmos DB container to another container within the Azure Cosmos DB account (could be in the same database or a different database).
18
-
* Move data from one Azure Cosmos DB account to another Azure Cosmos DB account (could be in the same region or a different regions, same subscription or a different one).
18
+
* Move data from one Azure Cosmos DB account to another Azure Cosmos DB account (could be in the same region or a different region, same subscription or a different one).
19
19
* Move data from a source such as Azure blob storage, a JSON file, Oracle database, Couchbase, DynamoDB to Azure Cosmos DB.
20
20
21
21
In order to support migration paths from the various sources to the different Azure Cosmos DB APIs, there are multiple solutions that provide specialized handling for each migration path. This document lists the available solutions and describes their advantages and limitations.
@@ -24,7 +24,7 @@ In order to support migration paths from the various sources to the different Az
24
24
25
25
The following factors determine the choice of the migration tool:
26
26
27
-
***Online vs offline migration**: Many migration tools provide a path to do a one-time migration only. This means that the applications accessing the database might experience a period of downtime. Some migration solutions provide a way to do a live migration where there is a replication pipeline set up between the source and the target.
27
+
***Online vs offline migration**: Many migration tools provide a path to do a one-time migration only. This means that the applications accessing the database might experience a period of downtime. Some migration solutions provide a way to do a live migration where there's a replication pipeline set up between the source and the target.
28
28
29
29
***Data source**: The existing data can be in various data sources like Oracle DB2, Datastax Cassanda, Azure SQL Database, PostgreSQL, etc. The data can also be in an existing Azure Cosmos DB account and the intent of migration can be to change the data model or repartition the data in a container with a different partition key.
30
30
@@ -37,35 +37,31 @@ The following factors determine the choice of the migration tool:
37
37
## Azure Cosmos DB API for NoSQL
38
38
39
39
If you need help with capacity planning, consider reading our [guide to estimating RU/s using Azure Cosmos DB capacity planner](estimate-ru-with-capacity-planner.md).
40
-
* If you are migrating from a vCores- or server-based platform and you need guidance on estimating request units, consider reading our [guide to estimating RU/s based on vCores](estimate-ru-with-capacity-planner.md).
41
-
42
-
>[!IMPORTANT]
43
-
> The [Custom Migration Service using ChangeFeed](https://github.com/Azure-Samples/azure-cosmosdb-live-data-migrator) is an open-source tool for live container migrations that implements change feed and bulk support. However, please note that the user interface application code for this tool is not supported or actively maintained by Microsoft. For Azure Cosmos DB API for NoSQL live container migrations, we recommend using the Spark Connector + Change Feed as illustrated in the [sample](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/cosmos/azure-cosmos-spark_3_2-12/Samples/DatabricksLiveContainerMigration) below. The [Spark Connector for Azure Cosmos DB](./nosql/quickstart-spark.md) is fully supported by Microsoft.
40
+
* If you're migrating from a vCores- or server-based platform and you need guidance on estimating request units, consider reading our [guide to estimating RU/s based on vCores](estimate-ru-with-capacity-planner.md).
|Offline|[Intra-account container copy](intra-account-container-copy.md)|Azure Cosmos DB for NoSQL|Azure Cosmos DB for NoSQL|• CLI-based; No set up needed. <br/>• Supports large datasets.|
48
45
|Offline|[Azure Data Factory](../data-factory/connector-azure-cosmos-db.md)|•JSON/CSV Files<br/>•Azure Cosmos DB for NoSQL<br/>•Azure Cosmos DB for MongoDB<br/>•MongoDB <br/>•SQL Server<br/>•Table Storage<br/>•Azure Blob Storage <br/> <br/>See the [Azure Data Factory](../data-factory/connector-overview.md) article for other supported sources.|•Azure Cosmos DB for NoSQL<br/>•Azure Cosmos DB for MongoDB<br/>•JSON Files <br/><br/> See the [Azure Data Factory](../data-factory/connector-overview.md) article for other supported targets. |• Easy to set up and supports multiple sources.<br/>• Makes use of the Azure Cosmos DB bulk executor library. <br/>• Suitable for large datasets. <br/>• Lack of checkpointing - It means that if an issue occurs during the course of migration, you need to restart the whole migration process.<br/>• Lack of a dead letter queue - It means that a few erroneous files can stop the entire migration process.|
49
46
|Offline|[Azure Cosmos DB Spark connector](./nosql/quickstart-spark.md)|Azure Cosmos DB for NoSQL. <br/><br/>You can use other sources with additional connectors from the Spark ecosystem.| Azure Cosmos DB for NoSQL. <br/><br/>You can use other targets with additional connectors from the Spark ecosystem.|• Makes use of the Azure Cosmos DB bulk executor library. <br/>• Suitable for large datasets. <br/>• Needs a custom Spark setup. <br/>• Spark is sensitive to schema inconsistencies and this can be a problem during migration. |
50
-
|Online|[Azure Cosmos DB Spark connector + Change Feed](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/cosmos/azure-cosmos-spark_3_2-12/Samples/DatabricksLiveContainerMigration)|Azure Cosmos DB for NoSQL. <br/><br/>Uses Azure Cosmos DB Change Feed to stream all historic data as well as live updates.| Azure Cosmos DB for NoSQL. <br/><br/>You can use other targets with additional connectors from the Spark ecosystem.|• Makes use of the Azure Cosmos DB bulk executor library. <br/>• Suitable for large datasets. <br/>• Needs a custom Spark setup. <br/>• Spark is sensitive to schema inconsistencies and this can be a problem during migration. |
47
+
|Online|[Azure Cosmos DB Spark connector + Change Feed sample](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/cosmos/azure-cosmos-spark_3_2-12/Samples/DatabricksLiveContainerMigration)|Azure Cosmos DB for NoSQL. <br/><br/>Uses Azure Cosmos DB Change Feed to stream all historic data as well as live updates.| Azure Cosmos DB for NoSQL. <br/><br/>You can use other targets with additional connectors from the Spark ecosystem.|• Makes use of the Azure Cosmos DB bulk executor library. <br/>• Suitable for large datasets. <br/>• Needs a custom Spark setup. <br/>• Spark is sensitive to schema inconsistencies and this can be a problem during migration. |
51
48
|Offline|[Custom tool with Azure Cosmos DB bulk executor library](migrate.md)| The source depends on your custom code | Azure Cosmos DB for NoSQL|• Provides checkpointing, dead-lettering capabilities which increases migration resiliency. <br/>• Suitable for very large datasets (10 TB+). <br/>• Requires custom setup of this tool running as an App Service. |
52
-
|Online|[Azure Cosmos DB Functions + ChangeFeed API](change-feed-functions.md)| Azure Cosmos DB for NoSQL | Azure Cosmos DB for NoSQL|• Easy to set up. <br/>• Works only if the source is an Azure Cosmos DB container. <br/>• Not suitable for large datasets. <br/>• Does not capture deletes from the source container. |
53
-
|Online|[Custom Migration Service using ChangeFeed](https://github.com/Azure-Samples/azure-cosmosdb-live-data-migrator)| Azure Cosmos DB for NoSQL | Azure Cosmos DB for NoSQL|• Provides progress tracking. <br/>• Works only if the source is an Azure Cosmos DB container. <br/>• Works for larger datasets as well.<br/>• Requires the user to set up an App Service to host the Change feed processor. <br/>• Does not capture deletes from the source container.|
49
+
|Online|[Azure Cosmos DB Functions + ChangeFeed API](change-feed-functions.md)| Azure Cosmos DB for NoSQL | Azure Cosmos DB for NoSQL|• Easy to set up. <br/>• Works only if the source is an Azure Cosmos DB container. <br/>• Not suitable for large datasets. <br/>• Doesn't capture deletes from the source container. |
54
50
|Online|[Striim](cosmosdb-sql-api-migrate-data-striim.md)|•Oracle <br/>•Apache Cassandra<br/><br/> See the [Striim website](https://www.striim.com/sources-and-targets/) for other supported sources. |•Azure Cosmos DB for NoSQL <br/>• Azure Cosmos DB for Cassandra<br/><br/> See the [Striim website](https://www.striim.com/sources-and-targets/) for other supported targets. |• Works with a large variety of sources like Oracle, DB2, SQL Server.<br/>• Easy to build ETL pipelines and provides a dashboard for monitoring. <br/>• Supports larger datasets. <br/>• Since this is a third-party tool, it needs to be purchased from the marketplace and installed in the user's environment.|
55
51
56
52
## Azure Cosmos DB API for MongoDB
57
53
58
54
Follow the [pre-migration guide](mongodb/pre-migration-steps.md) to plan your migration.
59
55
* If you need help with capacity planning, consider reading our [guide to estimating RU/s using Azure Cosmos DB capacity planner](estimate-ru-with-capacity-planner.md).
60
-
* If you are migrating from a vCores- or server-based platform and you need guidance on estimating request units, consider reading our [guide to estimating RU/s based on vCores](convert-vcore-to-request-unit.md).
56
+
* If you're migrating from a vCores- or server-based platform and you need guidance on estimating request units, consider reading our [guide to estimating RU/s based on vCores](convert-vcore-to-request-unit.md).
61
57
62
-
When you are ready to migrate, you can find detailed guidance on migration tools below
58
+
When you're ready to migrate, you can find detailed guidance on migration tools below
63
59
*[Offline migration using MongoDB native tools](mongodb/tutorial-mongotools-cosmos-db.md)
64
60
*[Offline migration using Azure database migration service (DMS)](../dms/tutorial-mongodb-cosmos-db.md)
65
61
*[Online migration using Azure database migration service (DMS)](../dms/tutorial-mongodb-cosmos-db-online.md)
66
62
*[Offline/online migration using Azure Databricks and Spark](mongodb/migrate-databricks.md)
67
63
68
-
Then, follow our [post-migration guide](mongodb/post-migration-optimization.md) to optimize your Azure Cosmos DB data estate once you have migrated.
64
+
Then, follow our [post-migration guide](mongodb/post-migration-optimization.md) to optimize your Azure Cosmos DB data estate once you've migrated.
69
65
70
66
A summary of migration pathways from your current solution to Azure Cosmso DB for MongoDB is provided below:
@@ -455,7 +455,7 @@ In the Visual Studio Code integrated terminal, change the directory to the ***&l
455
455
dotnet build
456
456
```
457
457
458
-
Open the file `Program.cs` and add a breakpoint.
458
+
Open the file `ModuleBackgroundService.cs` and add a breakpoint.
459
459
460
460
Navigate to the Visual Studio Code Debug view by selecting the debug icon from the menu on the left or by typing `Ctrl+Shift+D`. Select the debug configuration ***<your module name>* Local Debug (.NET Core)** from the dropdown.
461
461
@@ -521,7 +521,7 @@ On your development machine, you can start an IoT Edge simulator instead of inst
521
521
522
522
### Build and run container for debugging and debug in attach mode
523
523
524
-
1. Open your module file (`Program.cs`, `app.js`, `App.java`, or `<your module name>.cs`) and add a breakpoint.
524
+
1. Open your module file (`ModuleBackgroundService.cs`, `app.js`, `App.java`, or `<your module name>.cs`) and add a breakpoint.
525
525
526
526
1. In the Visual Studio Code Explorer view, right-click the `deployment.debug.template.json` file for your solution and then select **Build and Run IoT Edge solution in Simulator**. You can watch all the module container logs in the same window. You can also navigate to the Docker view to watch container status.
527
527
@@ -685,7 +685,7 @@ Open the module file for your development language and add a breakpoint:
685
685
686
686
# [C\#](#tab/csharp)
687
687
688
-
Add your breakpoint to the file `Program.cs`.
688
+
Add your breakpoint to the file `ModuleBackgroundService.cs`.
0 commit comments