You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/storage/blobs/storage-blob-change-feed-how-to.md
+86-67Lines changed: 86 additions & 67 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
---
2
2
title: Process change feed in Azure Blob Storage
3
3
titleSuffix: Azure Storage
4
-
description: Learn how to process change feed logs in a .NET client application
4
+
description: Learn how to process change feed transaction logs in a .NET client application using the Blobs Change Feed client library.
5
5
author: normesta
6
6
7
7
ms.author: normesta
8
-
ms.date: 03/03/2022
8
+
ms.date: 06/06/2024
9
9
ms.topic: article
10
10
ms.service: azure-blob-storage
11
11
ms.reviewer: sadodd
@@ -19,35 +19,66 @@ Change feed provides transaction logs of all the changes that occur to the blobs
19
19
20
20
To learn more about the change feed, see [Change feed in Azure Blob Storage](storage-blob-change-feed.md).
21
21
22
-
## Get the blob change feed processor library
22
+
## Set up your project
23
23
24
-
1. Open a command window (For example: Windows PowerShell).
25
-
2. From your project directory, install the [**Azure.Storage.Blobs.Changefeed** NuGet package](https://www.nuget.org/packages/Azure.Storage.Blobs.ChangeFeed/).
24
+
This section walks you through preparing a project to work with the Blobs Change Feed client library for .NET.
25
+
26
+
### Install packages
27
+
28
+
From your project directory, install the package for the [Azure Storage Blobs Change Feed client library for .NET](/dotnet/api/overview/azure/storage.blobs.changefeed-readme) using the `dotnet add package` command. In this example, we add the `--prerelease` flag to the command to install the latest preview version.
The code examples in this article also use the [Azure Blob Storage](/dotnet/api/azure.storage.blobs) and [Azure Identity](/dotnet/api/azure.identity) packages.
35
+
36
+
```console
37
+
dotnet add package Azure.Identity
38
+
dotnet add package Azure.Storage.Blobs
39
+
```
40
+
41
+
### Add `using` directives
42
+
43
+
Add the following `using` directives to your code file:
44
+
45
+
```csharp
46
+
usingAzure.Identity;
47
+
usingAzure.Storage.Blobs;
48
+
usingAzure.Storage.Blobs.ChangeFeed;
30
49
```
31
50
32
-
## Read records
51
+
### Create a client object
52
+
53
+
To connect the application to Blob Storage, create an instance of the `BlobServiceClient` class. The following example shows how to create a client object using `DefaultAzureCredential` for authorization. To learn more, see [Authorize access and connect to Blob Storage](storage-blob-dotnet-get-started.md#authorize-access-and-connect-to-blob-storage). To work with the change feed, you need Azure RBAC built-in role **Storage Blob Data Reader** or higher.
54
+
55
+
```csharp
56
+
// TODO: Replace <storage-account-name> with the name of your storage account
The client object is passed as a parameter to some of the methods shown in this article.
65
+
66
+
## Read records in the change feed
33
67
34
68
> [!NOTE]
35
69
> The change feed is an immutable and read-only entity in your storage account. Any number of applications can read and process the change feed simultaneously and independently at their own convenience. Records aren't removed from the change feed when an application reads them. The read or iteration state of each consuming reader is independent and maintained by your application only.
36
70
37
-
This example iterates through all records in the change feed, adds them to a list, and then returns that list to the caller.
71
+
The following code example iterates through all records in the change feed, adds them to a list, and then returns the list of change feed events:
You can choose to save your read position in the change feed, and then resume iterating through the records at a future time. You can save the read position by getting the change feed cursor. The cursor is a **string** and your application can save that string in any way that makes sense for your application's design (For example: to a file, or database).
110
+
You can choose to save your read position in the change feed, and then resume iterating through the records at a future time. You can save the read position by getting the change feed cursor. The cursor is a **string** and your application can save that string in any way that makes sense for your application's design, for example, to a file or database.
81
111
82
112
This example iterates through all records in the change feed, adds them to a list, and saves the cursor. The list and the cursor are returned to the caller.
// Update the change feed cursor. The cursor is not required to get each page of events,
109
-
// it is intended to be saved and used to resume iterating at a later date.
136
+
// Update the change feed cursor. The cursor is not required to get each page of events,
137
+
// it's intended to be saved and used to resume iterating at a later date.
110
138
cursor=enumerator.Current.ContinuationToken;
111
139
return (cursor, changeFeedEvents);
112
140
}
113
141
```
114
142
115
143
## Stream processing of records
116
144
117
-
You can choose to process change feed records as they are committed to the change feed. See [Specifications](storage-blob-change-feed.md#specifications). The change events are published to the change feed at a period of 60 seconds on average. We recommend that you poll for new changes with this period in mind when specifying your poll interval.
145
+
You can choose to process change feed records as they're committed to the change feed. See [Specifications](storage-blob-change-feed.md#specifications). The change events are published to the change feed at a period of 60 seconds on average. We recommend that you poll for new changes with this period in mind when specifying your poll interval.
118
146
119
-
This example periodically polls for changes. If change records exist, this code processes those records and saves change feed cursor. That way if the process is stopped and then started again, the application can use the cursor to resume processing records where it last left off. This example saves the cursor to a local application configuration file, but your application can save it in any form that makes the most sense for your scenario.
147
+
This example periodically polls for changes. If change records exist, this code processes those records and saves change feed cursor. That way if the process is stopped and then started again, the application can use the cursor to resume processing records where it last left off. This example saves the cursor to a local file for demonstration purposes, but your application can save it in any form that makes the most sense for your scenario.
You can read records that fall within a specific time range. This example iterates through all records in the change feed that fall between 3:00 PM on March 2 2020 and 2:00 AM on August 7 2020, adds them to a list, and then returns that list to the caller.
203
+
## Read records within a specific time range
182
204
183
-
### Selecting segments for a time range
205
+
You can read records that fall within a specific time range. This example iterates through all records in the change feed that fall within a specific date and time range, adds them to a list, and returns the list:
Copy file name to clipboardExpand all lines: articles/storage/blobs/storage-blob-dotnet-get-started.md
+5-4Lines changed: 5 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -58,13 +58,11 @@ To connect an application to Blob Storage, create an instance of the [BlobServic
58
58
59
59
To learn more about creating and managing client objects, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).
60
60
61
-
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS).
62
-
63
-
To learn more about each of these authorization mechanisms, see [Authorize access to data in Azure Storage](../common/authorize-data-access.md).
61
+
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS). For optimal security, Microsoft recommends using Microsoft Entra ID with managed identities to authorize requests against blob data. For more information, see [Authorize access to blobs using Microsoft Entra ID](authorize-access-azure-active-directory.md).
64
62
65
63
<aname='azure-ad'></a>
66
64
67
-
## [Microsoft Entra ID](#tab/azure-ad)
65
+
## [Microsoft Entra ID (recommended)](#tab/azure-ad)
68
66
69
67
To authorize with Microsoft Entra ID, you'll need to use a security principal. The type of security principal you need depends on where your application runs. Use this table as a guide.
70
68
@@ -118,6 +116,9 @@ To learn more about generating and managing SAS tokens, see the following articl
118
116
-[Create a user delegation SAS for a container with .NET](storage-blob-container-user-delegation-sas-create-dotnet.md)
119
117
-[Create a user delegation SAS for a blob with .NET](storage-blob-user-delegation-sas-create-dotnet.md)
120
118
119
+
> [!NOTE]
120
+
> For scenarios where shared access signatures (SAS) are used, Microsoft recommends using a user delegation SAS. A user delegation SAS is secured with Microsoft Entra credentials instead of the account key.
121
+
121
122
## [Account key](#tab/account-key)
122
123
123
124
Create a [StorageSharedKeyCredential](/dotnet/api/azure.storage.storagesharedkeycredential) by using the storage account name and account key. Then use that object to initialize a [BlobServiceClient](/dotnet/api/azure.storage.blobs.blobserviceclient).
Copy file name to clipboardExpand all lines: articles/storage/blobs/storage-blob-go-get-started.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -67,7 +67,7 @@ You can authorize a client object using a Microsoft Entra authorization token (r
67
67
68
68
<aname='azure-ad'></a>
69
69
70
-
## [Microsoft Entra ID](#tab/azure-ad)
70
+
## [Microsoft Entra ID (recommended)](#tab/azure-ad)
71
71
72
72
To authorize with Microsoft Entra ID, you need to use a [security principal](../../active-directory/develop/app-objects-and-service-principals.md). The following articles provide guidance on different authentication scenarios:
73
73
@@ -88,7 +88,7 @@ To use a shared access signature (SAS) token, append the token to the account UR
> A user delegation SAS offers superior security to a SAS that is signed with the storage account key. Microsoft recommends using a user delegation SAS when possible. For more information, see [Grant limited access to data with shared access signatures (SAS)](../common/storage-sas-overview.md).
91
+
> For scenarios where shared access signatures (SAS) are used, Microsoft recommends using a user delegation SAS. A user delegation SAS is secured with Microsoft Entra credentials instead of the account key. For more information, see [Grant limited access to data with shared access signatures (SAS)](../common/storage-sas-overview.md).
Copy file name to clipboardExpand all lines: articles/storage/blobs/storage-blob-java-get-started.md
+5-2Lines changed: 5 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -120,11 +120,11 @@ To connect an application to Blob Storage, create an instance of the [BlobServic
120
120
121
121
To learn more about creating and managing client objects, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).
122
122
123
-
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS).
123
+
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS). For optimal security, Microsoft recommends using Microsoft Entra ID with managed identities to authorize requests against blob data. For more information, see [Authorize access to blobs using Microsoft Entra ID](authorize-access-azure-active-directory.md).
124
124
125
125
<aname='azure-ad-recommended'></a>
126
126
127
-
## [Microsoft Entra ID (Recommended)](#tab/azure-ad)
127
+
## [Microsoft Entra ID (recommended)](#tab/azure-ad)
128
128
129
129
To authorize with Microsoft Entra ID, you'll need to use a [security principal](../../active-directory/develop/app-objects-and-service-principals.md). Which type of security principal you need depends on where your application runs. Use the following table as a guide:
130
130
@@ -160,6 +160,9 @@ To learn more about generating and managing SAS tokens, see the following articl
160
160
-[Create a user delegation SAS for a container with Java](storage-blob-container-user-delegation-sas-create-java.md)
161
161
-[Create a user delegation SAS for a blob with Java](storage-blob-user-delegation-sas-create-java.md)
162
162
163
+
> [!NOTE]
164
+
> For scenarios where shared access signatures (SAS) are used, Microsoft recommends using a user delegation SAS. A user delegation SAS is secured with Microsoft Entra credentials instead of the account key.
165
+
163
166
## [Account key](#tab/account-key)
164
167
165
168
Create a [StorageSharedKeyCredential](/java/api/com.azure.storage.common.storagesharedkeycredential) by using the storage account name and account key. Then use that object to initialize a [BlobServiceClient](/java/api/com.azure.storage.blob.blobserviceclient) object.
0 commit comments