Skip to content

Commit 4cac808

Browse files
authored
Merge pull request #277436 from pauljewellmsft/samples-auth
Updates to auth guidance
2 parents 394df35 + 03b0831 commit 4cac808

7 files changed

+144
-87
lines changed

articles/storage/blobs/storage-blob-change-feed-how-to.md

Lines changed: 86 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
---
22
title: Process change feed in Azure Blob Storage
33
titleSuffix: Azure Storage
4-
description: Learn how to process change feed logs in a .NET client application
4+
description: Learn how to process change feed transaction logs in a .NET client application using the Blobs Change Feed client library.
55
author: normesta
66

77
ms.author: normesta
8-
ms.date: 03/03/2022
8+
ms.date: 06/06/2024
99
ms.topic: article
1010
ms.service: azure-blob-storage
1111
ms.reviewer: sadodd
@@ -19,35 +19,66 @@ Change feed provides transaction logs of all the changes that occur to the blobs
1919

2020
To learn more about the change feed, see [Change feed in Azure Blob Storage](storage-blob-change-feed.md).
2121

22-
## Get the blob change feed processor library
22+
## Set up your project
2323

24-
1. Open a command window (For example: Windows PowerShell).
25-
2. From your project directory, install the [**Azure.Storage.Blobs.Changefeed** NuGet package](https://www.nuget.org/packages/Azure.Storage.Blobs.ChangeFeed/).
24+
This section walks you through preparing a project to work with the Blobs Change Feed client library for .NET.
25+
26+
### Install packages
27+
28+
From your project directory, install the package for the [Azure Storage Blobs Change Feed client library for .NET](/dotnet/api/overview/azure/storage.blobs.changefeed-readme) using the `dotnet add package` command. In this example, we add the `--prerelease` flag to the command to install the latest preview version.
2629

2730
```console
28-
dotnet add package Azure.Storage.Blobs --version 12.5.1
29-
dotnet add package Azure.Storage.Blobs.ChangeFeed --version 12.0.0-preview.4
31+
dotnet add package Azure.Storage.Blobs.ChangeFeed --prerelease
32+
```
33+
34+
The code examples in this article also use the [Azure Blob Storage](/dotnet/api/azure.storage.blobs) and [Azure Identity](/dotnet/api/azure.identity) packages.
35+
36+
```console
37+
dotnet add package Azure.Identity
38+
dotnet add package Azure.Storage.Blobs
39+
```
40+
41+
### Add `using` directives
42+
43+
Add the following `using` directives to your code file:
44+
45+
```csharp
46+
using Azure.Identity;
47+
using Azure.Storage.Blobs;
48+
using Azure.Storage.Blobs.ChangeFeed;
3049
```
3150

32-
## Read records
51+
### Create a client object
52+
53+
To connect the application to Blob Storage, create an instance of the `BlobServiceClient` class. The following example shows how to create a client object using `DefaultAzureCredential` for authorization. To learn more, see [Authorize access and connect to Blob Storage](storage-blob-dotnet-get-started.md#authorize-access-and-connect-to-blob-storage). To work with the change feed, you need Azure RBAC built-in role **Storage Blob Data Reader** or higher.
54+
55+
```csharp
56+
// TODO: Replace <storage-account-name> with the name of your storage account
57+
string accountName = "<storage-account-name>";
58+
59+
BlobServiceClient client = new(
60+
new Uri($"https://{accountName}.blob.core.windows.net"),
61+
new DefaultAzureCredential());
62+
```
63+
64+
The client object is passed as a parameter to some of the methods shown in this article.
65+
66+
## Read records in the change feed
3367

3468
> [!NOTE]
3569
> The change feed is an immutable and read-only entity in your storage account. Any number of applications can read and process the change feed simultaneously and independently at their own convenience. Records aren't removed from the change feed when an application reads them. The read or iteration state of each consuming reader is independent and maintained by your application only.
3670
37-
This example iterates through all records in the change feed, adds them to a list, and then returns that list to the caller.
71+
The following code example iterates through all records in the change feed, adds them to a list, and then returns the list of change feed events:
3872

3973
```csharp
40-
public async Task<List<BlobChangeFeedEvent>> ChangeFeedAsync(string connectionString)
74+
public async Task<List<BlobChangeFeedEvent>> ChangeFeedAsync(BlobServiceClient client)
4175
{
42-
// Get a new blob service client.
43-
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
76+
// Create a new BlobChangeFeedClient
77+
BlobChangeFeedClient changeFeedClient = client.GetChangeFeedClient();
4478

45-
// Get a new change feed client.
46-
BlobChangeFeedClient changeFeedClient = blobServiceClient.GetChangeFeedClient();
79+
List<BlobChangeFeedEvent> changeFeedEvents = [];
4780

48-
List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();
49-
50-
// Get all the events in the change feed.
81+
// Get all the events in the change feed
5182
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync())
5283
{
5384
changeFeedEvents.Add(changeFeedEvent);
@@ -57,43 +88,40 @@ public async Task<List<BlobChangeFeedEvent>> ChangeFeedAsync(string connectionSt
5788
}
5889
```
5990

60-
This example prints to the console a few values from each record in the list.
61-
91+
The following code example prints some values from the list of change feed events:
6292
```csharp
6393
public void showEventData(List<BlobChangeFeedEvent> changeFeedEvents)
6494
{
6595
foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedEvents)
6696
{
6797
string subject = changeFeedEvent.Subject;
6898
string eventType = changeFeedEvent.EventType.ToString();
69-
string api = changeFeedEvent.EventData.Api;
99+
BlobOperationName operationName = changeFeedEvent.EventData.BlobOperationName;
70100

71101
Console.WriteLine("Subject: " + subject + "\n" +
72102
"Event Type: " + eventType + "\n" +
73-
"Api: " + api);
103+
"Operation: " + operationName.ToString());
74104
}
75105
}
76106
```
77107

78108
## Resume reading records from a saved position
79109

80-
You can choose to save your read position in the change feed, and then resume iterating through the records at a future time. You can save the read position by getting the change feed cursor. The cursor is a **string** and your application can save that string in any way that makes sense for your application's design (For example: to a file, or database).
110+
You can choose to save your read position in the change feed, and then resume iterating through the records at a future time. You can save the read position by getting the change feed cursor. The cursor is a **string** and your application can save that string in any way that makes sense for your application's design, for example, to a file or database.
81111

82112
This example iterates through all records in the change feed, adds them to a list, and saves the cursor. The list and the cursor are returned to the caller.
83113

84114
```csharp
85-
public async Task<(string, List<BlobChangeFeedEvent>)> ChangeFeedResumeWithCursorAsync
86-
(string connectionString, string cursor)
115+
public async Task<(string, List<BlobChangeFeedEvent>)> ChangeFeedResumeWithCursorAsync(
116+
BlobServiceClient client,
117+
string cursor)
87118
{
88-
// Get a new blob service client.
89-
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
90-
91-
// Get a new change feed client.
92-
BlobChangeFeedClient changeFeedClient = blobServiceClient.GetChangeFeedClient();
119+
// Get a new change feed client
120+
BlobChangeFeedClient changeFeedClient = client.GetChangeFeedClient();
93121
List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();
94122

95123
IAsyncEnumerator<Page<BlobChangeFeedEvent>> enumerator = changeFeedClient
96-
.GetChangesAsync(continuation: cursor)
124+
.GetChangesAsync(continuationToken: cursor)
97125
.AsPages(pageSizeHint: 10)
98126
.GetAsyncEnumerator();
99127

@@ -102,38 +130,37 @@ public async Task<(string, List<BlobChangeFeedEvent>)> ChangeFeedResumeWithCurso
102130
foreach (BlobChangeFeedEvent changeFeedEvent in enumerator.Current.Values)
103131
{
104132

105-
changeFeedEvents.Add(changeFeedEvent);
133+
changeFeedEvents.Add(changeFeedEvent);
106134
}
107135

108-
// Update the change feed cursor. The cursor is not required to get each page of events,
109-
// it is intended to be saved and used to resume iterating at a later date.
136+
// Update the change feed cursor. The cursor is not required to get each page of events,
137+
// it's intended to be saved and used to resume iterating at a later date.
110138
cursor = enumerator.Current.ContinuationToken;
111139
return (cursor, changeFeedEvents);
112140
}
113141
```
114142

115143
## Stream processing of records
116144

117-
You can choose to process change feed records as they are committed to the change feed. See [Specifications](storage-blob-change-feed.md#specifications). The change events are published to the change feed at a period of 60 seconds on average. We recommend that you poll for new changes with this period in mind when specifying your poll interval.
145+
You can choose to process change feed records as they're committed to the change feed. See [Specifications](storage-blob-change-feed.md#specifications). The change events are published to the change feed at a period of 60 seconds on average. We recommend that you poll for new changes with this period in mind when specifying your poll interval.
118146

119-
This example periodically polls for changes. If change records exist, this code processes those records and saves change feed cursor. That way if the process is stopped and then started again, the application can use the cursor to resume processing records where it last left off. This example saves the cursor to a local application configuration file, but your application can save it in any form that makes the most sense for your scenario.
147+
This example periodically polls for changes. If change records exist, this code processes those records and saves change feed cursor. That way if the process is stopped and then started again, the application can use the cursor to resume processing records where it last left off. This example saves the cursor to a local file for demonstration purposes, but your application can save it in any form that makes the most sense for your scenario.
120148

121149
```csharp
122-
public async Task ChangeFeedStreamAsync
123-
(string connectionString, int waitTimeMs, string cursor)
150+
public async Task ChangeFeedStreamAsync(
151+
BlobServiceClient client,
152+
int waitTimeMs,
153+
string cursor)
124154
{
125-
// Get a new blob service client.
126-
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
127-
128-
// Get a new change feed client.
129-
BlobChangeFeedClient changeFeedClient = blobServiceClient.GetChangeFeedClient();
155+
// Get a new change feed client
156+
BlobChangeFeedClient changeFeedClient = client.GetChangeFeedClient();
130157

131158
while (true)
132159
{
133160
IAsyncEnumerator<Page<BlobChangeFeedEvent>> enumerator = changeFeedClient
134-
.GetChangesAsync(continuation: cursor).AsPages().GetAsyncEnumerator();
161+
.GetChangesAsync(continuationToken: cursor).AsPages().GetAsyncEnumerator();
135162

136-
while (true)
163+
while (true)
137164
{
138165
var result = await enumerator.MoveNextAsync();
139166

@@ -143,14 +170,14 @@ public async Task ChangeFeedStreamAsync
143170
{
144171
string subject = changeFeedEvent.Subject;
145172
string eventType = changeFeedEvent.EventType.ToString();
146-
string api = changeFeedEvent.EventData.Api;
173+
BlobOperationName operationName = changeFeedEvent.EventData.BlobOperationName;
147174

148175
Console.WriteLine("Subject: " + subject + "\n" +
149176
"Event Type: " + eventType + "\n" +
150-
"Api: " + api);
177+
"Operation: " + operationName.ToString());
151178
}
152179

153-
// helper method to save cursor.
180+
// Helper method to save cursor
154181
SaveCursor(enumerator.Current.ContinuationToken);
155182
}
156183
else
@@ -161,42 +188,34 @@ public async Task ChangeFeedStreamAsync
161188
}
162189
await Task.Delay(waitTimeMs);
163190
}
164-
165191
}
166192

167-
public void SaveCursor(string cursor)
193+
void SaveCursor(string cursor)
168194
{
169-
System.Configuration.Configuration config =
170-
ConfigurationManager.OpenExeConfiguration
171-
(ConfigurationUserLevel.None);
195+
// Specify the path to the file where you want to save the cursor
196+
string filePath = "path/to/cursor.txt";
172197

173-
config.AppSettings.Settings.Clear();
174-
config.AppSettings.Settings.Add("Cursor", cursor);
175-
config.Save(ConfigurationSaveMode.Modified);
198+
// Write the cursor value to the file
199+
File.WriteAllText(filePath, cursor);
176200
}
177201
```
178202

179-
## Reading records within a time range
180-
181-
You can read records that fall within a specific time range. This example iterates through all records in the change feed that fall between 3:00 PM on March 2 2020 and 2:00 AM on August 7 2020, adds them to a list, and then returns that list to the caller.
203+
## Read records within a specific time range
182204

183-
### Selecting segments for a time range
205+
You can read records that fall within a specific time range. This example iterates through all records in the change feed that fall within a specific date and time range, adds them to a list, and returns the list:
184206

185207
```csharp
186-
public async Task<List<BlobChangeFeedEvent>> ChangeFeedBetweenDatesAsync(string connectionString)
208+
async Task<List<BlobChangeFeedEvent>> ChangeFeedBetweenDatesAsync(BlobServiceClient client)
187209
{
188-
// Get a new blob service client.
189-
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
190-
191-
// Get a new change feed client.
192-
BlobChangeFeedClient changeFeedClient = blobServiceClient.GetChangeFeedClient();
210+
// Get a new change feed client
211+
BlobChangeFeedClient changeFeedClient = client.GetChangeFeedClient();
193212
List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();
194213

195214
// Create the start and end time. The change feed client will round start time down to
196215
// the nearest hour, and round endTime up to the next hour if you provide DateTimeOffsets
197216
// with minutes and seconds.
198-
DateTimeOffset startTime = new DateTimeOffset(2020, 3, 2, 15, 0, 0, TimeSpan.Zero);
199-
DateTimeOffset endTime = new DateTimeOffset(2020, 8, 7, 2, 0, 0, TimeSpan.Zero);
217+
DateTimeOffset startTime = new DateTimeOffset(2024, 3, 1, 0, 0, 0, TimeSpan.Zero);
218+
DateTimeOffset endTime = new DateTimeOffset(2024, 6, 1, 0, 0, 0, TimeSpan.Zero);
200219

201220
// You can also provide just a start or end time.
202221
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(

articles/storage/blobs/storage-blob-dotnet-get-started.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -58,13 +58,11 @@ To connect an application to Blob Storage, create an instance of the [BlobServic
5858

5959
To learn more about creating and managing client objects, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).
6060

61-
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS).
62-
63-
To learn more about each of these authorization mechanisms, see [Authorize access to data in Azure Storage](../common/authorize-data-access.md).
61+
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS). For optimal security, Microsoft recommends using Microsoft Entra ID with managed identities to authorize requests against blob data. For more information, see [Authorize access to blobs using Microsoft Entra ID](authorize-access-azure-active-directory.md).
6462

6563
<a name='azure-ad'></a>
6664

67-
## [Microsoft Entra ID](#tab/azure-ad)
65+
## [Microsoft Entra ID (recommended)](#tab/azure-ad)
6866

6967
To authorize with Microsoft Entra ID, you'll need to use a security principal. The type of security principal you need depends on where your application runs. Use this table as a guide.
7068

@@ -118,6 +116,9 @@ To learn more about generating and managing SAS tokens, see the following articl
118116
- [Create a user delegation SAS for a container with .NET](storage-blob-container-user-delegation-sas-create-dotnet.md)
119117
- [Create a user delegation SAS for a blob with .NET](storage-blob-user-delegation-sas-create-dotnet.md)
120118

119+
> [!NOTE]
120+
> For scenarios where shared access signatures (SAS) are used, Microsoft recommends using a user delegation SAS. A user delegation SAS is secured with Microsoft Entra credentials instead of the account key.
121+
121122
## [Account key](#tab/account-key)
122123

123124
Create a [StorageSharedKeyCredential](/dotnet/api/azure.storage.storagesharedkeycredential) by using the storage account name and account key. Then use that object to initialize a [BlobServiceClient](/dotnet/api/azure.storage.blobs.blobserviceclient).

articles/storage/blobs/storage-blob-go-get-started.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ You can authorize a client object using a Microsoft Entra authorization token (r
6767

6868
<a name='azure-ad'></a>
6969

70-
## [Microsoft Entra ID](#tab/azure-ad)
70+
## [Microsoft Entra ID (recommended)](#tab/azure-ad)
7171

7272
To authorize with Microsoft Entra ID, you need to use a [security principal](../../active-directory/develop/app-objects-and-service-principals.md). The following articles provide guidance on different authentication scenarios:
7373

@@ -88,7 +88,7 @@ To use a shared access signature (SAS) token, append the token to the account UR
8888
:::code language="go" source="~/blob-devguide-go/cmd/client-auth/client_auth.go" id="snippet_get_service_client_SAS":::
8989

9090
> [!NOTE]
91-
> A user delegation SAS offers superior security to a SAS that is signed with the storage account key. Microsoft recommends using a user delegation SAS when possible. For more information, see [Grant limited access to data with shared access signatures (SAS)](../common/storage-sas-overview.md).
91+
> For scenarios where shared access signatures (SAS) are used, Microsoft recommends using a user delegation SAS. A user delegation SAS is secured with Microsoft Entra credentials instead of the account key. For more information, see [Grant limited access to data with shared access signatures (SAS)](../common/storage-sas-overview.md).
9292
9393
## [Account key](#tab/account-key)
9494

articles/storage/blobs/storage-blob-java-get-started.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,11 +120,11 @@ To connect an application to Blob Storage, create an instance of the [BlobServic
120120

121121
To learn more about creating and managing client objects, see [Create and manage client objects that interact with data resources](storage-blob-client-management.md).
122122

123-
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS).
123+
You can authorize a `BlobServiceClient` object by using a Microsoft Entra authorization token, an account access key, or a shared access signature (SAS). For optimal security, Microsoft recommends using Microsoft Entra ID with managed identities to authorize requests against blob data. For more information, see [Authorize access to blobs using Microsoft Entra ID](authorize-access-azure-active-directory.md).
124124

125125
<a name='azure-ad-recommended'></a>
126126

127-
## [Microsoft Entra ID (Recommended)](#tab/azure-ad)
127+
## [Microsoft Entra ID (recommended)](#tab/azure-ad)
128128

129129
To authorize with Microsoft Entra ID, you'll need to use a [security principal](../../active-directory/develop/app-objects-and-service-principals.md). Which type of security principal you need depends on where your application runs. Use the following table as a guide:
130130

@@ -160,6 +160,9 @@ To learn more about generating and managing SAS tokens, see the following articl
160160
- [Create a user delegation SAS for a container with Java](storage-blob-container-user-delegation-sas-create-java.md)
161161
- [Create a user delegation SAS for a blob with Java](storage-blob-user-delegation-sas-create-java.md)
162162

163+
> [!NOTE]
164+
> For scenarios where shared access signatures (SAS) are used, Microsoft recommends using a user delegation SAS. A user delegation SAS is secured with Microsoft Entra credentials instead of the account key.
165+
163166
## [Account key](#tab/account-key)
164167

165168
Create a [StorageSharedKeyCredential](/java/api/com.azure.storage.common.storagesharedkeycredential) by using the storage account name and account key. Then use that object to initialize a [BlobServiceClient](/java/api/com.azure.storage.blob.blobserviceclient) object.

0 commit comments

Comments
 (0)