|
| 1 | +--- |
| 2 | +title: Migrate from the bulk executor library to the Azure Cosmos DB .NET V3 SDK |
| 3 | +description: Learn how to migrate your application from using the bulk executor library to the Azure Cosmos DB SDK V3 |
| 4 | +author: ealsur |
| 5 | +ms.service: cosmos-db |
| 6 | +ms.topic: conceptual |
| 7 | +ms.date: 03/24/2020 |
| 8 | +ms.author: maquaran |
| 9 | +--- |
| 10 | + |
| 11 | +# Migrate from the bulk executor library to the Azure Cosmos DB .NET V3 SDK |
| 12 | + |
| 13 | +This article describes the required steps to migrate an existing application's code that uses the [.NET bulk executor library](bulk-executor-dot-net.md) to the [bulk support](tutorial-sql-api-dotnet-bulk-import.md) feature in the latest version of the .NET SDK (also referred as .NET V3 SDK) and it's useful if your current production code relies on the response types provided by the bulk executor library. |
| 14 | + |
| 15 | +## Enable bulk support |
| 16 | + |
| 17 | +Make sure you are enabling bulk support on the `CosmosClient` instance through the [AllowBulkExecution](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmos.cosmosclientoptions.allowbulkexecution) configuration flag: |
| 18 | + |
| 19 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="CreateClient"::: |
| 20 | + |
| 21 | +## Create Tasks for each operation |
| 22 | + |
| 23 | +Bulk support in the .NET SDK version 3 works by leveraging the [Task Parallel Library](https://docs.microsoft.com/dotnet/standard/parallel-programming/task-parallel-library-tpl) and grouping operations that occur concurrently. This means that there is no single method that will take your list of documents or operations as an input parameter, but rather, you need to create a Task for each operation you want to execute in bulk. |
| 24 | + |
| 25 | +If our initial input is a list of items where each item has, for example, this schema: |
| 26 | + |
| 27 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="Model"::: |
| 28 | + |
| 29 | +If you want to do bulk *import* (similar to using [BulkExecutor.BulkImportAsync](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkexecutor.bulkimportasync)) it means you need to have concurrent calls to `CreateItemAsync` with each item value. For example: |
| 30 | + |
| 31 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="BulkImport"::: |
| 32 | + |
| 33 | +If you want to do bulk *update* (similar to using [BulkExecutor.BulkUpdateAsync](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkexecutor.bulkupdateasync)), it means you need to have concurrent calls to `ReplaceItemAsync` after updating the item value. For example: |
| 34 | + |
| 35 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="BulkUpdate"::: |
| 36 | + |
| 37 | +And if you want to do bulk *delete* (similar to using [BulkExecutor.BulkDeleteAsync](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkexecutor.bulkdeleteasync)), it means you need to have concurrent calls to `DeleteItemAsync`, with the `id` and partition key of each item. For example: |
| 38 | + |
| 39 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="BulkDelete"::: |
| 40 | + |
| 41 | +## Capture task result state |
| 42 | + |
| 43 | +In the previous code examples, we are creating a concurrent list of Tasks, and on each of them, calling `CaptureOperationResponse`. This is an extension that lets us maintain a *similar response schema* as BulkExecutor, by capturing any errors and tracking the [request units usage](request-units.md). |
| 44 | + |
| 45 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="CaptureOperationResult"::: |
| 46 | + |
| 47 | +Where the `OperationResponse` is declared as: |
| 48 | + |
| 49 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="OperationResponse"::: |
| 50 | + |
| 51 | +## Execute operations concurrently |
| 52 | + |
| 53 | +With the list of Tasks defined, all we need to do is wait until they are all completed, which would define the *scope* of our bulk operation. This is easily done by: |
| 54 | + |
| 55 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="WhenAll"::: |
| 56 | + |
| 57 | +## Capture statistics |
| 58 | + |
| 59 | +The code above not only waits until all operations are complete, but it also captures the time spent and calculates the required statistics. These statistics are similar to that of the bulk executor library's [BulkImportResponse](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.bulkimport.bulkimportresponse). |
| 60 | + |
| 61 | + :::code language="csharp" source="~/samples-cosmosdb-dotnet-v3/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration/Program.cs" id="ResponseType"::: |
| 62 | + |
| 63 | +The `BulkOperationResponse` will contain: |
| 64 | + |
| 65 | +1. The total time taken to process the list of operations through bulk support. |
| 66 | +1. The number of successful operations. |
| 67 | +1. The total of request units consumed. |
| 68 | +1. If any failures happened, a list of tuples containing the captured Exception and the associated item for logging and identification purposes. |
| 69 | + |
| 70 | +## Performance improvements |
| 71 | + |
| 72 | +As with other operations with the .NET SDK, leveraging the stream APIs will be better performance-wise because it would avoid any unnecessary serialization, but it is only possible if the nature of the data we work with matches that of a stream of bytes (for example, file streams). In those cases, using the `CreateItemStreamAsync`, `ReplaceItemStreamAsync`, or `DeleteItemStreamAsync` APIs and working with `ResponseMessage` (instead of `ItemResponse`) will increase the throughput that can be achieved. |
| 73 | + |
| 74 | +## Additional resources |
| 75 | + |
| 76 | +* [Azure Cosmos DB SDK](sql-api-sdk-dotnet.md) |
| 77 | +* [Complete migration source code on GitHub](https://github.com/Azure/azure-cosmos-dotnet-v3/tree/master/Microsoft.Azure.Cosmos.Samples/Usage/BulkExecutorMigration) |
| 78 | +* [Additional bulk samples on GitHub](https://github.com/Azure/azure-cosmos-dotnet-v3/tree/master/Microsoft.Azure.Cosmos.Samples/Usage/BulkSupport) |
| 79 | + |
| 80 | +## Next steps |
| 81 | + |
| 82 | +You can now proceed to learn more about change feed processor in the following articles: |
| 83 | + |
| 84 | +* [Overview of change feed processor](change-feed-processor.md) |
| 85 | +* [Using the change feed estimator](how-to-use-change-feed-estimator.md) |
| 86 | +* [Change feed processor start time](how-to-configure-change-feed-start-time.md) |
0 commit comments