Skip to content

Commit 21cb675

Browse files
Merge pull request #281747 from HeidiSteen/heidist-freshness
[azure search] refresh synonym map doc
2 parents 1943996 + 2e001b5 commit 21cb675

File tree

2 files changed

+113
-39
lines changed

2 files changed

+113
-39
lines changed

articles/search/search-synonyms.md

Lines changed: 109 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,40 @@
11
---
22
title: Synonyms for query expansion
33
titleSuffix: Azure AI Search
4-
description: Create a synonym map to expand the scope of a search query over an Azure AI Search index. Scope is broadened to include equivalent terms you provide in the synonym map.
4+
description: Create a synonym map to expand the scope of a search query over an Azure AI Search index. The query can search on equivalent terms provided in the synonym map, even if the query doesn't explicitly include the term.
55

66
manager: nitinme
77
author: HeidiSteen
88
ms.author: heidist
99
ms.service: cognitive-search
1010
ms.custom:
1111
- ignite-2023
12-
ms.topic: conceptual
13-
ms.date: 04/22/2024
12+
ms.topic: how-to
13+
ms.date: 07/22/2024
1414
---
1515

1616
# Synonyms in Azure AI Search
1717

18-
On a search service, synonym maps are a global resource that associate equivalent terms, expanding the scope of a query without the user having to actually provide the term. For example, assuming "dog", "canine", and "puppy" are mapped synonyms, a query on "canine" will match on a document containing "dog".
18+
On a search service, a synonym map associates equivalent terms, expanding the scope of a query without the user having to actually provide the term. For example, assuming "dog", "canine", and "puppy" are mapped synonyms, a query on "canine" matches on a document containing "dog". You might create multiple synonym maps for different languages, such as English and French versions, or lexicons if your content includes technical jargon, slang, or obscure terminology.
1919

20-
## Create synonyms
20+
Some key points about synonym maps:
2121

22-
A synonym map is an asset that can be created once and used by many indexes. The [service tier](search-limits-quotas-capacity.md#synonym-limits) determines how many synonym maps you can create, ranging from three synonym maps for Free and Basic tiers, up to 20 for the Standard tiers.
22+
- A synonym map is a top-level resource that can be created once and used by many indexes.
23+
- A synonym map applies to string fields.
24+
- You can create and assign a synonym map at any time with no disruption to indexing or queries.
25+
- Your [service tier](search-limits-quotas-capacity.md#synonym-limits) sets the limits on how many synonym maps you can create.
26+
- Your search service can have multiple synonym maps, but within an index, a field definition can only have one synonym map assignment.
2327

24-
You might create multiple synonym maps for different languages, such as English and French versions, or lexicons if your content includes technical jargon, slang, or obscure terminology. Although you can create multiple synonym maps in your search service, within an index, a field definition can only have one synonym map assignment.
28+
## Create a synonym map
2529

2630
A synonym map consists of name, format, and rules that function as synonym map entries. The only format that is supported is `solr`, and the `solr` format determines rule construction.
2731

32+
To create a synonym map, do so programmatically. The portal doesn't support synonym map definitions.
33+
34+
### [REST](#tab/rest)
35+
36+
Use the [Create Synonym Map (REST API)](/rest/api/searchservice/create-synonym-map) to create a synonym map.
37+
2838
```http
2939
POST /synonymmaps?api-version=2023-11-01
3040
{
@@ -36,25 +46,35 @@ POST /synonymmaps?api-version=2023-11-01
3646
}
3747
```
3848

39-
To create a synonym map, do so programmatically (the portal doesn't support synonym map definitions):
49+
### [.NET](#tab/dotnet)
50+
51+
Use the [SynonymMap class (.NET)](/dotnet/api/azure.search.documents.indexes.models.synonymmap) and [Create a synonym map(Azure SDK sample)](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample02_Service.md#create-a-synonym-map) to create the map.
52+
53+
### [Python](#tab/python)
54+
55+
Use the [SynonymMap class (Python)](/python/api/azure-search-documents/azure.search.documents.indexes.models.synonymmap) to create the map.
56+
57+
### [Java](#tab/java)
58+
59+
Use the [SynonymMap class (Java)](/java/api/com.azure.search.documents.indexes.models.synonymmap) to create the map.
4060

41-
+ [Create Synonym Map (REST API)](/rest/api/searchservice/create-synonym-map). This reference is the most descriptive.
42-
+ [SynonymMap class (.NET)](/dotnet/api/azure.search.documents.indexes.models.synonymmap) and [Create a synonym map(Azure SDK sample)](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample02_Service.md#create-a-synonym-map)
43-
+ [SynonymMap class (Python)](/python/api/azure-search-documents/azure.search.documents.indexes.models.synonymmap)
44-
+ [SynonymMap interface (JavaScript)](/javascript/api/@azure/search-documents/synonymmap)
45-
+ [SynonymMap class (Java)](/java/api/com.azure.search.documents.indexes.models.synonymmap)
61+
### [JavaScript](#tab/javascript)
4662

47-
## Define rules
63+
Use the [SynonymMap interface (JavaScript)](/javascript/api/@azure/search-documents/synonymmap) to create the map.
64+
65+
---
66+
67+
### Define rules
4868

4969
Mapping rules adhere to the open-source synonym filter specification of Apache Solr, described in this document: [SynonymFilter](https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-SynonymFilter). The `solr` format supports two kinds of rules:
5070

51-
+ equivalency (where terms are equal substitutes in the query)
71+
- equivalency (where terms are equal substitutes in the query)
5272

53-
+ explicit mappings (where terms are mapped to one explicit term prior to querying)
73+
- explicit mappings (where terms are mapped to one explicit term)
5474

55-
Each rule must be delimited by the new line character (`\n`). You can define up to 5,000 rules per synonym map in a free service and 20,000 rules per map in other tiers. Each rule can have up to 20 expansions (or items in a rule). For more information, see [Synonym limits](search-limits-quotas-capacity.md#synonym-limits).
75+
Each rule is delimited by the new line character (`\n`). You can define up to 5,000 rules per synonym map in a free service and 20,000 rules per map in other tiers. Each rule can have up to 20 expansions (or items in a rule). For more information, see [Synonym limits](search-limits-quotas-capacity.md#synonym-limits).
5676

57-
Query parsers automatically lower-case any upper or mixed case terms, but if you want to preserve special characters in the string, such as a comma or dash, add the appropriate escape characters when creating the synonym map.
77+
Query parsers automatically lower-case any upper or mixed case terms. To preserve special characters in the string, such as a comma or dash, add the appropriate escape characters when creating the synonym map.
5878

5979
### Equivalency rules
6080

@@ -89,12 +109,12 @@ In the explicit case, a query for `Washington`, `Wash.` or `WA` is rewritten as
89109

90110
### Escaping special characters
91111

92-
In full text search, synonyms are analyzed during query processing just like any other query term, which means that rules around reserved and special characters apply to the terms in your synonym map. The list of characters that requires escaping varies between the simple syntax and full syntax:
112+
Synonyms are analyzed during query processing just like any other query term, which means that rules for reserved and special characters apply to the terms in your synonym map. The list of characters that requires escaping varies between the simple syntax and full syntax:
93113

94-
+ [simple syntax](query-simple-syntax.md) `+ | " ( ) ' \`
95-
+ [full syntax](query-lucene-syntax.md) `+ - & | ! ( ) { } [ ] ^ " ~ * ? : \ /`
114+
- [simple syntax](query-simple-syntax.md) `+ | " ( ) ' \`
115+
- [full syntax](query-lucene-syntax.md) `+ - & | ! ( ) { } [ ] ^ " ~ * ? : \ /`
96116

97-
Recall that if you need to preserve characters that would otherwise be discarded by the default analyzer during indexing, you should substitute an analyzer that preserves them. Some choices include Microsoft natural [language analyzers](index-add-language-analyzers.md), which preserves hyphenated words, or a custom analyzer for more complex patterns. For more information, see [Partial terms, patterns, and special characters](search-query-partial-matching.md).
117+
To preserve characters that the default analyzer discards, substitute an analyzer that preserves them. Some choices include Microsoft natural [language analyzers](index-add-language-analyzers.md), which preserves hyphenated words, or a custom analyzer for more complex patterns. For more information, see [Partial terms, patterns, and special characters](search-query-partial-matching.md).
98118

99119
The following example shows an example of how to escape a character with a backslash:
100120

@@ -105,7 +125,7 @@ The following example shows an example of how to escape a character with a backs
105125
}
106126
```
107127

108-
Since the backslash is itself a special character in other languages like JSON and C#, you'll probably need to double-escape it. For example, the JSON sent to the REST API for the above synonym map would look like this:
128+
Since the backslash is itself a special character in other languages like JSON and C#, you probably need to double-escape it. Here's an example in JSON:
109129

110130
```json
111131
{
@@ -114,15 +134,25 @@ Since the backslash is itself a special character in other languages like JSON a
114134
}
115135
```
116136

117-
## Upload and manage synonym maps
137+
## Manage synonym maps
118138

119-
As mentioned previously, you can create or update a synonym map without disrupting query and indexing workloads. A synonym map is a standalone object (like indexes or data sources), and as long as no field is using it, updates won't cause indexing or queries to fail. However, once you add a synonym map to a field definition, if you then delete a synonym map, any query that includes the fields in question will fail with a 404 error.
139+
You can update a synonym map without disrupting query and indexing workloads. However, once you add a synonym map to a field, if you then delete a synonym map, any query that includes the fields in question fail with a 404 error.
120140

121-
Creating, updating, and deleting a synonym map is always a whole-document operation, meaning that you can't update or delete parts of the synonym map incrementally. Updating even a single rule requires a reload.
141+
Creating, updating, and deleting a synonym map is always a whole-document operation. You can't update or delete parts of the synonym map incrementally. Updating even a single rule requires a reload.
122142

123143
## Assign synonyms to fields
124144

125-
After uploading a synonym map, you can enable the synonyms on fields of the type `Edm.String` or `Collection(Edm.String)`, on fields having `"searchable":true`. As noted, a field definition can use only one synonym map.
145+
After you create the synonym map, assign it to a field in your index. To assign synonym maps, do so programmatically. The portal doesn't support synonym map field associations.
146+
147+
- A field must be of type `Edm.String` or `Collection(Edm.String)`
148+
- A field must have `"searchable":true`
149+
- A field can have only one synonym map
150+
151+
If the synonym map exists on the search service, it's used on the next query, with no reindexing or rebuild required.
152+
153+
### [REST](#tab/rest-assign)
154+
155+
Use the [Create or Update Index (REST API)](/rest/api/searchservice/indexes/create-or-update) to modify a field definition.
126156

127157
```http
128158
POST /indexes?api-version=2023-11-01
@@ -150,15 +180,63 @@ POST /indexes?api-version=2023-11-01
150180
}
151181
```
152182

183+
### [**.NET SDK**](#tab/dotnet-assign)
184+
185+
Use the [**SearchIndexClient**](/dotnet/api/azure.search.documents.indexes.searchindexclient) to update an index. Provide the whole index definition and include the new parameters for synonym map assignments.
186+
187+
In this example, the "country" field has a synonymMapName property.
188+
189+
```csharp
190+
// Update anindex
191+
string indexName = "hotels";
192+
SearchIndex index = new SearchIndex(indexName)
193+
{
194+
Fields =
195+
{
196+
new SimpleField("hotelId", SearchFieldDataType.String) { IsKey = true, IsFilterable = true, IsSortable = true },
197+
new SearchableField("hotelName") { IsFilterable = true, IsSortable = true },
198+
new SearchableField("description") { AnalyzerName = LexicalAnalyzerName.EnLucene },
199+
new SearchableField("descriptionFr") { AnalyzerName = LexicalAnalyzerName.FrLucene }
200+
new ComplexField("address")
201+
{
202+
Fields =
203+
{
204+
new SearchableField("streetAddress"),
205+
new SearchableField("city") { IsFilterable = true, IsSortable = true, IsFacetable = true },
206+
new SearchableField("stateProvince") { IsFilterable = true, IsSortable = true, IsFacetable = true },
207+
new SearchableField("country") { SynonymMapNames = new[] { synonymMapName }, IsFilterable = true, IsSortable = true, IsFacetable = true },
208+
new SearchableField("postalCode") { IsFilterable = true, IsSortable = true, IsFacetable = true }
209+
}
210+
}
211+
}
212+
};
213+
214+
await indexClient.CreateIndexAsync(index);
215+
```
216+
217+
For more examples, see[azure-search-dotnet-samples/quickstart/v11/](https://github.com/Azure-Samples/azure-search-dotnet-samples/tree/main/quickstart/v11).
218+
219+
### [**Other SDKs**](#tab/other-sdks-assign)
220+
221+
You can use any supported SDK to update a search index. All of them provide a **SearchIndexClient** that has methods for updating indexes.
222+
223+
| Azure SDK | Client | Examples |
224+
|-----------|--------|----------|
225+
| Java | [SearchIndexClient](/java/api/com.azure.search.documents.indexes.searchindexclient) | [CreateIndexExample.java](https://github.com/Azure/azure-sdk-for-java/blob/azure-search-documents_11.1.3/sdk/search/azure-search-documents/src/samples/java/com/azure/search/documents/indexes/CreateIndexExample.java) |
226+
| JavaScript | [SearchIndexClient](/javascript/api/@azure/search-documents/searchindexclient) | [Indexes](https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/search/search-documents/samples/v11/javascript) |
227+
| Python | [SearchIndexClient](/python/api/azure-search-documents/azure.search.documents.indexes.searchindexclient) | [sample_index_crud_operations.py](https://github.com/Azure/azure-sdk-for-python/blob/7cd31ac01fed9c790cec71de438af9c45cb45821/sdk/search/azure-search-documents/samples/sample_index_crud_operations.py) |
228+
229+
---
230+
153231
## Query on equivalent or mapped fields
154232

155-
Adding synonyms doesn't impose new requirements on query construction. You can issue term and phrase queries just as you did before the addition of synonyms. The only difference is that if a query term exists in the synonym map, the query engine will either expand or rewrite the term or phrase, depending on the rule.
233+
A synonym field assignment doesn't change how you write queries. After the synonym map assignment, the only difference is that if a query term exists in the synonym map, the search engine either expands or rewrites the term or phrase, depending on the rule.
156234

157235
## How synonyms are used during query execution
158236

159-
Synonyms are a query expansion technique that supplements the contents of an index with equivalent terms, but only for fields that have a synonym assignment. If a field-scoped query *excludes* a synonym-enabled field, you won't see matches from the synonym map.
237+
Synonyms are a query expansion technique that supplements the contents of an index with equivalent terms, but only for fields that have a synonym assignment. If a field-scoped query *excludes* a synonym-enabled field, you don't see matches from the synonym map.
160238

161-
For synonym-enabled fields, synonyms are subject to the same text analysis as the associated field. For example, if a field is analyzed using the standard Lucene analyzer, synonym terms will also be subject to the standard Lucene analyzer at query time. If you want to preserve punctuation, such as periods or dashes, in the synonym term, apply a content-preserving analyzer on the field.
239+
For synonym-enabled fields, synonyms are subject to the same text analysis as the associated field. For example, if a field is analyzed using the standard Lucene analyzer, synonym terms are also subject to the standard Lucene analyzer at query time. If you want to preserve punctuation, such as periods or dashes, in the synonym term, apply a content-preserving analyzer on the field.
162240

163241
Internally, the synonyms feature rewrites the original query with synonyms with the OR operator. For this reason, hit highlighting and scoring profiles treat the original term and synonyms as equivalent.
164242

@@ -168,7 +246,7 @@ Synonym expansions don't apply to wildcard search terms; prefix, fuzzy, and rege
168246

169247
If you need to do a single query that applies synonym expansion and wildcard, regex, or fuzzy searches, you can combine the queries using the OR syntax. For example, to combine synonyms with wildcards for simple query syntax, the term would be `<query> | <query>*`.
170248

171-
If you have an existing index in a development (non-production) environment, experiment with a small dictionary to see how the addition of synonyms changes the search experience, including impact on scoring profiles, hit highlighting, and suggestions.
249+
If you have an existing index in a development (nonproduction) environment, experiment with a small dictionary to see how the addition of synonyms changes the search experience, including impact on scoring profiles, hit highlighting, and suggestions.
172250

173251
## Next steps
174252

articles/search/semantic-how-to-query-request.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ ms.service: cognitive-search
1010
ms.custom:
1111
- ignite-2023
1212
ms.topic: how-to
13-
ms.date: 06/13/2024
13+
ms.date: 07/24/2024
1414
---
1515

1616
# Add semantic ranking to queries in Azure AI Search
@@ -43,11 +43,9 @@ You can use any of the following tools and SDKs to build a query that uses seman
4343

4444
## Avoid features that bypass relevance scoring
4545

46-
Several query capabilities in Azure AI Search bypass relevance scoring or are otherwise incompatible with semantic ranking. If your query logic includes the following features, you can't semantically rank your results:
46+
A few query capabilities bypass relevance scoring, which makes them incompatible with semantic ranking. If your query logic includes the following features, you can't semantically rank your results:
4747

48-
+ A query with `search=*` or an empty search string, such as pure filter-only query, won't work because there's nothing to measure semantic relevance against. The query must provide terms or phrases that can be assessed during processing.
49-
50-
+ A query composed in the [full Lucene syntax](query-lucene-syntax.md) (`queryType=full`) is incompatible with semantic ranking (`queryType=semantic`). The semantic model doesn't support the full Lucene syntax.
48+
+ A query with `search=*` or an empty search string, such as pure filter-only query, won't work because there's nothing to measure semantic relevance against and so the search scores are zero. The query must provide terms or phrases that can be evaluated during processing.
5149

5250
+ Sorting (orderBy clauses) on specific fields overrides search scores and a semantic score. Given that the semantic score is supposed to provide the ranking, adding an orderby clause results in an HTTP 400 error if you apply semantic ranking over ordered results.
5351

@@ -116,9 +114,7 @@ The following example in this section uses the [hotels-sample-index](search-get-
116114
117115
1. Set "queryType" to "semantic".
118116
119-
In other queries, the "queryType" is used to specify the query parser. In semantic ranking, it's set to "semantic". For the "search" field, you can specify queries that conform to the [simple syntax](query-simple-syntax.md).
120-
121-
1. Set "search" to a full text search query based on the [simple syntax](query-simple-syntax.md). Semantic ranking is an extension of full text search, so while this parameter isn't required, you won't get an expected outcome if it's null.
117+
1. Set "search" to a full text search query. Your search string can support either the [simple syntax](query-simple-syntax.md) or [full Lucene syntax](query-lucene-syntax.md). Semantic ranking is an extension of full text search, so while "search" isn't required, you won't get an expected outcome if it's an empty search (`"search": "*"`).
122118
123119
1. Set "semanticConfiguration" to a [predefined semantic configuration](semantic-how-to-configure.md) that's embedded in your index.
124120

0 commit comments

Comments
 (0)