Skip to content

Commit 5263777

Browse files
committed
adding doc
1 parent 35f1e29 commit 5263777

File tree

2 files changed

+98
-5
lines changed

2 files changed

+98
-5
lines changed

articles/cosmos-db/bulk-executor-dot-net.md

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,14 @@ ms.service: cosmos-db
66
ms.subservice: cosmosdb-sql
77
ms.devlang: dotnet
88
ms.topic: conceptual
9-
ms.date: 09/01/2019
9+
ms.date: 03/23/2020
1010
ms.author: ramkris
1111
ms.reviewer: sngun
1212
---
1313

14+
> [!NOTE]
15+
> For new applications, we have added **bulk support** directly into the [.NET SDK version 3.x](tutorial-sql-api-dotnet-bulk-import.md). This page is maintained for applications using the previous .NET SDK 2.x.
16+
1417
# Use the bulk executor .NET library to perform bulk operations in Azure Cosmos DB
1518

1619
This tutorial provides instructions on using the bulk executor .NET library to import and update documents to an Azure Cosmos container. To learn about the bulk executor library and how it helps you leverage massive throughput and storage, see the [bulk executor library overview](bulk-executor-overview.md) article. In this tutorial, you will see a sample .NET application that bulk imports randomly generated documents into an Azure Cosmos container. After importing, it shows you how you can bulk update the imported data by specifying patches as operations to perform on specific document fields.
@@ -19,7 +22,7 @@ Currently, bulk executor library is supported by the Azure Cosmos DB SQL API and
1922

2023
## Prerequisites
2124

22-
* If you dont already have Visual Studio 2019 installed, you can download and use the [Visual Studio 2019 Community Edition](https://www.visualstudio.com/downloads/). Make sure that you enable "Azure development" during the Visual Studio setup.
25+
* If you don't already have Visual Studio 2019 installed, you can download and use the [Visual Studio 2019 Community Edition](https://www.visualstudio.com/downloads/). Make sure that you enable "Azure development" during the Visual Studio setup.
2326

2427
* If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?ref=microsoft.com&utm_source=microsoft.com&utm_medium=docs&utm_campaign=visualstudio) before you begin.
2528

@@ -35,15 +38,15 @@ Now let's switch to working with code by downloading a sample .NET application f
3538
git clone https://github.com/Azure/azure-cosmosdb-bulkexecutor-dotnet-getting-started.git
3639
```
3740

38-
The cloned repository contains two samples "BulkImportSample" and "BulkUpdateSample". You can open either of the sample applications, update the connection strings in App.config file with your Azure Cosmos DB accounts connection strings, build the solution, and run it.
41+
The cloned repository contains two samples "BulkImportSample" and "BulkUpdateSample". You can open either of the sample applications, update the connection strings in App.config file with your Azure Cosmos DB account's connection strings, build the solution, and run it.
3942

4043
The "BulkImportSample" application generates random documents and bulk imports them to your Azure Cosmos account. The "BulkUpdateSample" application bulk updates the imported documents by specifying patches as operations to perform on specific document fields. In the next sections, you will review the code in each of these sample apps.
4144

4245
## Bulk import data to an Azure Cosmos account
4346

4447
1. Navigate to the "BulkImportSample" folder and open the "BulkImportSample.sln" file.
4548

46-
2. The Azure Cosmos DBs connection strings are retrieved from the App.config file as shown in the following code:
49+
2. The Azure Cosmos DB's connection strings are retrieved from the App.config file as shown in the following code:
4750

4851
```csharp
4952
private static readonly string EndpointUrl = ConfigurationManager.AppSettings["EndPointUrl"];
@@ -161,7 +164,7 @@ You can update existing documents by using the BulkUpdateAsync API. In this exam
161164
|NumberOfDocumentsUpdated (long) | The number of documents that were successfully updated out of the total documents supplied to the bulk update API call. |
162165
|TotalRequestUnitsConsumed (double) | The total request units (RUs) consumed by the bulk update API call. |
163166
|TotalTimeTaken (TimeSpan) | The total time taken by the bulk update API call to complete the execution. |
164-
167+
165168
## Performance tips
166169

167170
Consider the following points for better performance when using the bulk executor library:
@@ -193,6 +196,10 @@ Consider the following points for better performance when using the bulk executo
193196
</system.diagnostics>
194197
```
195198

199+
## Migration to .NET SDK version 3
200+
201+
[.NET SDK version 3](tutorial-sql-api-dotnet-bulk-import.md) **includes bulk support** and does not require any external library. If you are currently using the bulk executor library and planning to migrate to bulk support on the newer SDK, follow the steps in our [migration guide](how-to-migrate-from-bulk-executor-library.md).
202+
196203
## Next steps
197204

198205
* To learn about the Nuget package details and the release notes, see the [bulk executor SDK details](sql-api-sdk-bulk-executor-dot-net.md).
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
title: Migrate from the bulk executor library to the Azure Cosmos DB .NET V3 SDK
3+
description: Learn how to migrate your application from using the bulk executor library to the Azure Cosmos DB SDK V3
4+
author: ealsur
5+
ms.service: cosmos-db
6+
ms.topic: conceptual
7+
ms.date: 03/24/2020
8+
ms.author: maquaran
9+
---
10+
11+
# Migrate from the bulk executor library to the Azure Cosmos DB .NET V3 SDK
12+
13+
This article describes the required steps to migrate an existing application's code that uses the [.NET bulk executor library](bulk-executor-dot-net.md) to the [bulk support](tutorial-sql-api-dotnet-bulk-import.md) feature in the latest version of the .NET SDK (also referred as .NET V3 SDK) and it's useful if your current production code relies on the response types provided by the bulk executor library.
14+
15+
## Enable bulk support
16+
17+
Make sure you are enabling bulk support on the `CosmosClient` instance through the [AllowBulkExecution](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmos.cosmosclientoptions.allowbulkexecution) configuration flag:
18+
19+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="CreateClient":::
20+
21+
## Create Tasks for each operation
22+
23+
Bulk support in the .NET SDK version 3 works by leveraging the [Task Parallel Library](https://docs.microsoft.com/dotnet/standard/parallel-programming/task-parallel-library-tpl) and grouping operations that occur concurrently. This means that there is no single method that will take your list of documents or operations as an input parameter, but rather, you need to create a Task for each operation you want to execute in bulk.
24+
25+
If our initial input is a list of items where each item has, for example, this schema:
26+
27+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="Model":::
28+
29+
If you want to do bulk *import* (similar to using [BulkExecutor.BulkImportAsync](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkexecutor.bulkimportasync)) it means you need to have concurrent calls to `CreateItemAsync` with each item value. For example:
30+
31+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="BulkImport":::
32+
33+
If you want to do bulk *update* (similar to using [BulkExecutor.BulkUpdateAsync](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkexecutor.bulkupdateasync)), it means you need to have concurrent calls to `ReplaceItemAsync` after updating the item value. For example:
34+
35+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="BulkUpdate":::
36+
37+
And if you want to do bulk *delete* (similar to using [BulkExecutor.BulkDeleteAsync](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkexecutor.bulkdeleteasync)), it means you need to have concurrent calls to `DeleteItemAsync`, with the `id` and partition key of each item. For example:
38+
39+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="BulkDelete":::
40+
41+
## Capture task result state
42+
43+
In the previous code examples, we are creating a concurrent list of Tasks, and on each of them, calling `CaptureOperationResponse`. This is an extension that lets us maintain a *similar response schema* as BulkExecutor, by capturing any errors and tracking the [request units usage](request-units.md).
44+
45+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="CaptureOperationResult":::
46+
47+
Where the `OperationResponse` is declared as:
48+
49+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="OperationResponse":::
50+
51+
## Execute operations concurrently
52+
53+
With the list of Tasks defined, all we need to do is wait until they are all completed, which would define the *scope* of our bulk operation. This is easily done by:
54+
55+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="WhenAll":::
56+
57+
## Capture statistics
58+
59+
The code above not only waits until all operations are complete, but it also captures the time spent and calculates the required statistics. These statistics are similar to that of the bulk executor library's [BulkImportResponse](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkimport.bulkimportresponse).
60+
61+
:::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="ResponseType":::
62+
63+
The `BulkOperationResponse` will contain:
64+
65+
1. The total time taken to process the list of operations through bulk support.
66+
1. The number of successful operations.
67+
1. The total of request units consumed.
68+
1. If any failures happened, a list of tuples containing the captured Exception and the associated item for logging and identification purposes.
69+
70+
## Performance improvements
71+
72+
As with other operations with the .NET SDK, leveraging the stream APIs will be better performance-wise because it would avoid any unnecessary serialization, but it is only possible if the nature of the data we work with matches that of a stream of bytes (for example, file streams). In those cases, using the `CreateItemStreamAsync`, `ReplaceItemStreamAsync`, or `DeleteItemStreamAsync` APIs and working with `ResponseMessage` (instead of `ItemResponse`) will increase the throughput that can be achieved.
73+
74+
## Additional resources
75+
76+
* [Azure Cosmos DB SDK](sql-api-sdk-dotnet.md)
77+
* [Complete migration source code on GitHub](https://github.com/Azure/azure-cosmos-dotnet-v3/tree/master/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration)
78+
* [Additional bulk samples on GitHub](https://github.com/Azure/azure-cosmos-dotnet-v3/tree/master/Microsoft.Azure.Cosmos.Samples/Usage/BulkSupport)
79+
80+
## Next steps
81+
82+
You can now proceed to learn more about change feed processor in the following articles:
83+
84+
* [Overview of change feed processor](change-feed-processor.md)
85+
* [Using the change feed estimator](how-to-use-change-feed-estimator.md)
86+
* [Change feed processor start time](how-to-configure-change-feed-start-time.md)

0 commit comments

Comments
 (0)