You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ideally, you want to use a single Azure Data Factory (ADF) copy activity to query SQL data as the source and write the output directly to Azure Cosmos DB sink as proper JSON objects. Currently, it is not possible to perform the needed JSON transformation in one copy activity. If we try to copy the results of the above query into an Azure Cosmos DB SQL API container, we will see the OrderDetails field as a string property of our document, instead of the expected JSON array.
@@ -60,7 +60,7 @@ We can work around this current limitation in one of the following ways:
60
60
61
61
Let’s look at these approaches in more detail:
62
62
63
-
## Azure Data Factory with two copy activities
63
+
## Azure Data Factory
64
64
65
65
Although we cannot embed OrderDetails as a JSON-array in the destination Cosmos DB document, we can work around the issue by using two separate Copy Activities.
66
66
@@ -241,7 +241,7 @@ writeConfig = {
241
241
}
242
242
```
243
243
244
-
Then, we will query the source Database (in this case SQL Server) for both the order and order detail records, putting the results into Spark Dataframes. We will also create a list containing all the order ids, and a Thread pool for parallel operations:
244
+
Then, we will query the source Database (in this case SQL Server) for both the order and order detail records, putting the results into Spark Dataframes. We will also create a list containing all the order IDs, and a Thread pool for parallel operations:
Then, create a function for writing Orders into the target SQL API collection. This function will filter all order details for the given order id, convert them into a JSON array, and insert the array into a JSON document that we will write into the target SQL API Collection for that order:
277
+
Then, create a function for writing Orders into the target SQL API collection. This function will filter all order details for the given order ID, convert them into a JSON array, and insert the array into a JSON document that we will write into the target SQL API Collection for that order:
There are a significant number of use cases where transactions in the database can contain many parent-child relationships. These relationships can grow significantly over time, and prove difficult to manage. Forms of [hierarchical databases](https://en.wikipedia.org/wiki/Hierarchical_database_model) did emerge during the 1980s, but were not popular due to inefficiency in storage. They also lost traction as [Ted Codd’s relational model](https://en.wikipedia.org/wiki/Relational_model) became the de facto standard used by virtually all mainstream database management systems.
33
33
@@ -37,7 +37,7 @@ The emergence of [object oriented design](https://en.wikipedia.org/wiki/Object-o
Ironically, given their name, relational databases present a less than optimal solution for modeling deep and complex relationships. The reason for this is that relationships between entities do not actually exist in a relational database. They need to be computed at runtime, with complex relationships requiring cartesian joins in order to allow mapping using queries. As a result, operations become exponentially more expensive in terms of computation as relationships increase. In some cases, a relational database attempting to manage such entities will become unusable.
43
43
@@ -63,7 +63,7 @@ The [microservices](https://en.wikipedia.org/wiki/Microservices) pattern has gro
63
63
* a JavaScript engine and [query API](https://docs.microsoft.com/azure/cosmos-db/javascript-query-api) built into the database.
64
64
* a state-of-the-art [change feed](https://docs.microsoft.com/azure/cosmos-db/change-feed) which clients can subscribe to in order to get notified of modifications to a container.
65
65
66
-
## Some challenges with NoSQL Databases
66
+
## Some challenges with NoSQL databases
67
67
68
68
Although there are some clear advantages when implementing NoSQL databases, there are also some challenges that you may want to take into consideration. These may not be present to the same degree when working with the relational model:
0 commit comments