Skip to content

Commit 16340fa

Browse files
v-dirichardsv-dirichards
authored
Merge pull request #273289 from HeidiSteen/heidist-freshness
[azure search] Concurrency doc refresh, replaced code example
2 parents d340a06 + 5085ece commit 16340fa

File tree

1 file changed

+102
-133
lines changed

1 file changed

+102
-133
lines changed

‎articles/search/search-howto-concurrency.md

Lines changed: 102 additions & 133 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,22 @@
11
---
2-
title: How to manage concurrent writes to resources
2+
title: Manage concurrent writes
33
titleSuffix: Azure AI Search
44
description: Use optimistic concurrency to avoid mid-air collisions on updates or deletes to Azure AI Search indexes, indexers, data sources.
55

66
manager: nitinme
77
author: HeidiSteen
88
ms.author: heidist
99
ms.service: cognitive-search
10-
ms.topic: conceptual
11-
ms.date: 01/26/2021
10+
ms.topic: how-to
11+
ms.date: 04/23/2024
1212
ms.custom:
1313
- devx-track-csharp
1414
- ignite-2023
1515
---
16-
# How to manage concurrency in Azure AI Search
1716

18-
When managing Azure AI Search resources such as indexes and data sources, it's important to update resources safely, especially if resources are accessed concurrently by different components of your application. When two clients concurrently update a resource without coordination, race conditions are possible. To prevent this, Azure AI Search offers an *optimistic concurrency model*. There are no locks on a resource. Instead, there is an ETag for every resource that identifies the resource version so that you can formulate requests that avoid accidental overwrites.
17+
# Manage concurrency in Azure AI Search
1918

20-
> [!Tip]
21-
> Conceptual code in a [sample C# solution](https://github.com/Azure-Samples/search-dotnet-getting-started/tree/master/DotNetETagsExplainer) explains how concurrency control works in Azure AI Search. The code creates conditions that invoke concurrency control. Reading the [code fragment below](#samplecode) might be sufficient for most developers, but if you want to run it, edit appsettings.json to add the service name and an admin api-key. Given a service URL of `http://myservice.search.windows.net`, the service name is `myservice`.
19+
When managing Azure AI Search resources such as indexes and data sources, it's important to update resources safely, especially if resources are accessed concurrently by different components of your application. When two clients concurrently update a resource without coordination, race conditions are possible. To prevent this, Azure AI Search uses an *optimistic concurrency model*. There are no locks on a resource. Instead, there's an ETag for every resource that identifies the resource version so that you can formulate requests that avoid accidental overwrites.
2220

2321
## How it works
2422

@@ -28,147 +26,125 @@ All resources have an [*entity tag (ETag)*](https://en.wikipedia.org/wiki/HTTP_E
2826

2927
+ The REST API uses an [ETag](/rest/api/searchservice/common-http-request-and-response-headers-used-in-azure-search) on the request header.
3028

31-
+ The .NET SDK sets the ETag through an accessCondition object, setting the [If-Match | If-Match-None header](/rest/api/searchservice/common-http-request-and-response-headers-used-in-azure-search) on the resource. Objects that use ETags, such as [SynonymMap.ETag](/dotnet/api/azure.search.documents.indexes.models.synonymmap.etag) and [SearchIndex.ETag](/dotnet/api/azure.search.documents.indexes.models.searchindex.etag), have an accessCondition object.
29+
+ The Azure SDK for .NET sets the ETag through an accessCondition object, setting the [If-Match | If-Match-None header](/rest/api/searchservice/common-http-request-and-response-headers-used-in-azure-search) on the resource. Objects that use ETags, such as [SynonymMap.ETag](/dotnet/api/azure.search.documents.indexes.models.synonymmap.etag) and [SearchIndex.ETag](/dotnet/api/azure.search.documents.indexes.models.searchindex.etag), have an accessCondition object.
3230

33-
Every time you update a resource, its ETag changes automatically. When you implement concurrency management, all you're doing is putting a precondition on the update request that requires the remote resource to have the same ETag as the copy of the resource that you modified on the client. If a concurrent process has changed the remote resource already, the ETag will not match the precondition and the request will fail with HTTP 412. If you're using the .NET SDK, this manifests as a `CloudException` where the `IsAccessConditionFailed()` extension method returns true.
31+
Every time you update a resource, its ETag changes automatically. When you implement concurrency management, all you're doing is putting a precondition on the update request that requires the remote resource to have the same ETag as the copy of the resource that you modified on the client. If another process changes the remote resource, the ETag doesn't match the precondition and the request fails with HTTP 412. If you're using the .NET SDK, this failure manifests as an exception where the `IsAccessConditionFailed()` extension method returns true.
3432

3533
> [!Note]
36-
> There is only one mechanism for concurrency. It's always used regardless of which API is used for resource updates.
34+
> There is only one mechanism for concurrency. It's always used regardless of which API or SDK is used for resource updates.
3735
38-
<a name="samplecode"></a>
3936
## Use cases and sample code
4037

41-
The following code demonstrates accessCondition checks for key update operations:
42-
43-
+ Fail an update if the resource no longer exists
44-
+ Fail an update if the resource version changes
45-
46-
### Sample code from [DotNetETagsExplainer program](https://github.com/Azure-Samples/search-dotnet-getting-started/tree/master/DotNetETagsExplainer)
38+
The following code demonstrates optimistic concurrency for an update operation. It fails the second update because the object's ETag is changed by a previous update. More specifically, when the ETag in the request header no longer matches the ETag of the object, the search service return a status 400 bad request message, and the update fails.
4739

4840
```csharp
49-
class Program
41+
using Azure;
42+
using Azure.Search.Documents;
43+
using Azure.Search.Documents.Indexes;
44+
using Azure.Search.Documents.Indexes.Models;
45+
using System;
46+
using System.Net;
47+
using System.Threading.Tasks;
48+
49+
namespace AzureSearch.SDKHowTo
5050
{
51-
// This sample shows how ETags work by performing conditional updates and deletes
52-
// on an Azure AI Search index.
53-
static void Main(string[] args)
51+
class Program
5452
{
55-
IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
56-
IConfigurationRoot configuration = builder.Build();
57-
58-
SearchServiceClient serviceClient = CreateSearchServiceClient(configuration);
59-
60-
Console.WriteLine("Deleting index...\n");
61-
DeleteTestIndexIfExists(serviceClient);
62-
63-
// Every top-level resource in Azure AI Search has an associated ETag that keeps track of which version
64-
// of the resource you're working on. When you first create a resource such as an index, its ETag is
65-
// empty.
66-
Index index = DefineTestIndex();
67-
Console.WriteLine(
68-
$"Test index hasn't been created yet, so its ETag should be blank. ETag: '{index.ETag}'");
69-
70-
// Once the resource exists in Azure AI Search, its ETag will be populated. Make sure to use the object
71-
// returned by the SearchServiceClient! Otherwise, you will still have the old object with the
72-
// blank ETag.
73-
Console.WriteLine("Creating index...\n");
74-
index = serviceClient.Indexes.Create(index);
75-
76-
Console.WriteLine($"Test index created; Its ETag should be populated. ETag: '{index.ETag}'");
77-
78-
// ETags let you do some useful things you couldn't do otherwise. For example, by using an If-Match
79-
// condition, we can update an index using CreateOrUpdate and be guaranteed that the update will only
80-
// succeed if the index already exists.
81-
index.Fields.Add(new Field("name", AnalyzerName.EnMicrosoft));
82-
index =
83-
serviceClient.Indexes.CreateOrUpdate(
84-
index,
85-
accessCondition: AccessCondition.GenerateIfExistsCondition());
86-
87-
Console.WriteLine(
88-
$"Test index updated; Its ETag should have changed since it was created. ETag: '{index.ETag}'");
89-
90-
// More importantly, ETags protect you from concurrent updates to the same resource. If another
91-
// client tries to update the resource, it will fail as long as all clients are using the right
92-
// access conditions.
93-
Index indexForClient1 = index;
94-
Index indexForClient2 = serviceClient.Indexes.Get("test");
95-
96-
Console.WriteLine("Simulating concurrent update. To start, both clients see the same ETag.");
97-
Console.WriteLine($"Client 1 ETag: '{indexForClient1.ETag}' Client 2 ETag: '{indexForClient2.ETag}'");
98-
99-
// Client 1 successfully updates the index.
100-
indexForClient1.Fields.Add(new Field("a", DataType.Int32));
101-
indexForClient1 =
102-
serviceClient.Indexes.CreateOrUpdate(
103-
indexForClient1,
104-
accessCondition: AccessCondition.IfNotChanged(indexForClient1));
105-
106-
Console.WriteLine($"Test index updated by client 1; ETag: '{indexForClient1.ETag}'");
107-
108-
// Client 2 tries to update the index, but fails, thanks to the ETag check.
109-
try
110-
{
111-
indexForClient2.Fields.Add(new Field("b", DataType.Boolean));
112-
serviceClient.Indexes.CreateOrUpdate(
113-
indexForClient2,
114-
accessCondition: AccessCondition.IfNotChanged(indexForClient2));
115-
116-
Console.WriteLine("Whoops; This shouldn't happen");
117-
Environment.Exit(1);
118-
}
119-
catch (CloudException e) when (e.IsAccessConditionFailed())
53+
// This sample shows how ETags work by performing conditional updates and deletes
54+
// on an Azure Search index.
55+
static void Main(string[] args)
12056
{
121-
Console.WriteLine("Client 2 failed to update the index, as expected.");
57+
string serviceName = "PLACEHOLDER FOR YOUR SEARCH SERVICE NAME";
58+
string apiKey = "PLACEHOLDER FOR YOUR SEARCH SERVICE ADMIN API KEY";
59+
60+
// Create a SearchIndexClient to send create/delete index commands
61+
Uri serviceEndpoint = new Uri($"https://{serviceName}.search.windows.net/");
62+
AzureKeyCredential credential = new AzureKeyCredential(apiKey);
63+
SearchIndexClient adminClient = new SearchIndexClient(serviceEndpoint, credential);
64+
65+
// Delete index if it exists
66+
Console.WriteLine("Check for index and delete if it already exists...\n");
67+
DeleteTestIndexIfExists(adminClient);
68+
69+
// Every top-level resource in Azure Search has an associated ETag that keeps track of which version
70+
// of the resource you're working on. When you first create a resource such as an index, its ETag is
71+
// empty.
72+
SearchIndex index = DefineTestIndex();
73+
74+
Console.WriteLine(
75+
$"Test searchIndex hasn't been created yet, so its ETag should be blank. ETag: '{index.ETag}'");
76+
77+
// Once the resource exists in Azure Search, its ETag is populated. Make sure to use the object
78+
// returned by the SearchIndexClient. Otherwise, you will still have the old object with the
79+
// blank ETag.
80+
Console.WriteLine("Creating index...\n");
81+
index = adminClient.CreateIndex(index);
82+
Console.WriteLine($"Test index created; Its ETag should be populated. ETag: '{index.ETag}'");
83+
84+
85+
// ETags prevent concurrent updates to the same resource. If another
86+
// client tries to update the resource, it will fail as long as all clients are using the right
87+
// access conditions.
88+
SearchIndex indexForClientA = index;
89+
SearchIndex indexForClientB = adminClient.GetIndex("test-idx");
90+
91+
Console.WriteLine("Simulating concurrent update. To start, clients A and B see the same ETag.");
92+
Console.WriteLine($"ClientA ETag: '{indexForClientA.ETag}' ClientB ETag: '{indexForClientB.ETag}'");
93+
94+
// indexForClientA successfully updates the index.
95+
indexForClientA.Fields.Add(new SearchField("a", SearchFieldDataType.Int32));
96+
indexForClientA = adminClient.CreateOrUpdateIndex(indexForClientA);
97+
98+
Console.WriteLine($"Client A updates test-idx by adding a new field. The new ETag for test-idx is: '{indexForClientA.ETag}'");
99+
100+
// indexForClientB tries to update the index, but fails due to the ETag check.
101+
try
102+
{
103+
indexForClientB.Fields.Add(new SearchField("b", SearchFieldDataType.Boolean));
104+
adminClient.CreateOrUpdateIndex(indexForClientB);
105+
106+
Console.WriteLine("Whoops; This shouldn't happen");
107+
Environment.Exit(1);
108+
}
109+
catch (RequestFailedException e) when (e.Status == 400)
110+
{
111+
Console.WriteLine("Client B failed to update the index, as expected.");
112+
}
113+
114+
// Uncomment the next line to remove test-idx
115+
//adminClient.DeleteIndex("test-idx");
116+
Console.WriteLine("Complete. Press any key to end application...\n");
117+
Console.ReadKey();
122118
}
123119

124-
// You can also use access conditions with Delete operations. For example, you can implement an
125-
// atomic version of the DeleteTestIndexIfExists method from this sample like this:
126-
Console.WriteLine("Deleting index...\n");
127-
serviceClient.Indexes.Delete("test", accessCondition: AccessCondition.GenerateIfExistsCondition());
128-
129-
// This is slightly better than using the Exists method since it makes only one round trip to
130-
// Azure AI Search instead of potentially two. It also avoids an extra Delete request in cases where
131-
// the resource is deleted concurrently, but this doesn't matter much since resource deletion in
132-
// Azure AI Search is idempotent.
133-
134-
// And we're done! Bye!
135-
Console.WriteLine("Complete. Press any key to end application...\n");
136-
Console.ReadKey();
137-
}
138-
139-
private static SearchServiceClient CreateSearchServiceClient(IConfigurationRoot configuration)
140-
{
141-
string searchServiceName = configuration["SearchServiceName"];
142-
string adminApiKey = configuration["SearchServiceAdminApiKey"];
143-
144-
SearchServiceClient serviceClient =
145-
new SearchServiceClient(searchServiceName, new SearchCredentials(adminApiKey));
146-
return serviceClient;
147-
}
148120

149-
private static void DeleteTestIndexIfExists(SearchServiceClient serviceClient)
150-
{
151-
if (serviceClient.Indexes.Exists("test"))
121+
private static void DeleteTestIndexIfExists(SearchIndexClient adminClient)
152122
{
153-
serviceClient.Indexes.Delete("test");
123+
try
124+
{
125+
if (adminClient.GetIndex("test-idx") != null)
126+
{
127+
adminClient.DeleteIndex("test-idx");
128+
}
129+
}
130+
catch (RequestFailedException e) when (e.Status == 404)
131+
{
132+
//if an exception occurred and status is "Not Found", this is working as expected
133+
Console.WriteLine("Failed to find index and this is because it's not there.");
134+
}
154135
}
155-
}
156136

157-
private static Index DefineTestIndex() =>
158-
new Index()
159-
{
160-
Name = "test",
161-
Fields = new[] { new Field("id", DataType.String) { IsKey = true } }
162-
};
137+
private static SearchIndex DefineTestIndex() =>
138+
new SearchIndex("test-idx", new[] { new SearchField("id", SearchFieldDataType.String) { IsKey = true } });
163139
}
164140
}
165141
```
166142

167143
## Design pattern
168144

169-
A design pattern for implementing optimistic concurrency should include a loop that retries the access condition check, a test for the access condition, and optionally retrieves an updated resource before attempting to re-apply the changes.
145+
A design pattern for implementing optimistic concurrency should include a loop that retries the access condition check, a test for the access condition, and optionally retrieves an updated resource before attempting to reapply the changes.
170146

171-
This code snippet illustrates the addition of a synonymMap to an index that already exists. This code is from the [Synonym C# example for Azure AI Search](https://github.com/Azure-Samples/search-dotnet-getting-started/tree/v10/DotNetHowToSynonyms).
147+
This code snippet illustrates the addition of a synonymMap to an index that already exists.
172148

173149
The snippet gets the "hotels" index, checks the object version on an update operation, throws an exception if the condition fails, and then retries the operation (up to three times), starting with index retrieval from the server to get the latest version.
174150

@@ -190,7 +166,7 @@ private static void EnableSynonymsInHotelsIndexSafely(SearchServiceClient servic
190166
Console.WriteLine("Updated the index successfully.\n");
191167
break;
192168
}
193-
catch (CloudException e) when (e.IsAccessConditionFailed())
169+
catch (Exception e) when (e.IsAccessConditionFailed())
194170
{
195171
Console.WriteLine($"Index update failed : {e.Message}. Attempt({i}/{MaxNumTries}).\n");
196172
}
@@ -205,15 +181,8 @@ private static Index AddSynonymMapsToFields(Index index)
205181
}
206182
```
207183

208-
## Next steps
209-
210-
Try modifying other samples to exercise ETags or AccessCondition objects.
211-
212-
+ [search-dotnet-getting-started on GitHub](https://github.com/Azure-Samples/search-dotnet-getting-started). This repository includes the "DotNetEtagsExplainer" project.
213-
214-
+ [azure-search-dotnet-samples on GitHub](https://github.com/Azure-Samples/azure-search-dotnet-samples) contains additional C# samples.
215-
216184
## See also
217185

186+
+ [ETag Struct](/dotnet/api/azure.etag?view=azure-dotnet)
218187
+ [Common HTTP request and response headers](/rest/api/searchservice/common-http-request-and-response-headers-used-in-azure-search)
219188
+ [HTTP status codes](/rest/api/searchservice/http-status-codes)

0 commit comments

Comments
 (0)