You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: Create a synonym map to expand the scope of a search query over an Azure AI Search index. Scope is broadened to include equivalent terms you provide in the synonym map.
4
+
description: Create a synonym map to expand the scope of a search query over an Azure AI Search index. The query can search on equivalent terms provided in the synonym map, even if the query doesn't explicitly include the term.
5
5
6
6
manager: nitinme
7
7
author: HeidiSteen
8
8
ms.author: heidist
9
9
ms.service: cognitive-search
10
10
ms.custom:
11
11
- ignite-2023
12
-
ms.topic: conceptual
13
-
ms.date: 04/22/2024
12
+
ms.topic: how-to
13
+
ms.date: 07/22/2024
14
14
---
15
15
16
16
# Synonyms in Azure AI Search
17
17
18
-
On a search service, synonym maps are a global resource that associate equivalent terms, expanding the scope of a query without the user having to actually provide the term. For example, assuming "dog", "canine", and "puppy" are mapped synonyms, a query on "canine" will match on a document containing "dog".
18
+
On a search service, a synonym map associates equivalent terms, expanding the scope of a query without the user having to actually provide the term. For example, assuming "dog", "canine", and "puppy" are mapped synonyms, a query on "canine" matches on a document containing "dog". You might create multiple synonym maps for different languages, such as English and French versions, or lexicons if your content includes technical jargon, slang, or obscure terminology.
19
19
20
-
## Create synonyms
20
+
Some key points about synonym maps:
21
21
22
-
A synonym map is an asset that can be created once and used by many indexes. The [service tier](search-limits-quotas-capacity.md#synonym-limits) determines how many synonym maps you can create, ranging from three synonym maps for Free and Basic tiers, up to 20 for the Standard tiers.
22
+
- A synonym map is a top-level resource that can be created once and used by many indexes.
23
+
- A synonym map applies to string fields.
24
+
- You can create and assign a synonym map at any time with no disruption to indexing or queries.
25
+
- Your [service tier](search-limits-quotas-capacity.md#synonym-limits) sets the limits on how many synonym maps you can create.
26
+
- Your search service can have multiple synonym maps, but within an index, a field definition can only have one synonym map assignment.
23
27
24
-
You might create multiple synonym maps for different languages, such as English and French versions, or lexicons if your content includes technical jargon, slang, or obscure terminology. Although you can create multiple synonym maps in your search service, within an index, a field definition can only have one synonym map assignment.
28
+
## Create a synonym map
25
29
26
30
A synonym map consists of name, format, and rules that function as synonym map entries. The only format that is supported is `solr`, and the `solr` format determines rule construction.
27
31
32
+
To create a synonym map, do so programmatically. The portal doesn't support synonym map definitions.
33
+
34
+
### [REST](#tab/rest)
35
+
36
+
Use the [Create Synonym Map (REST API)](/rest/api/searchservice/create-synonym-map) to create a synonym map.
37
+
28
38
```http
29
39
POST /synonymmaps?api-version=2023-11-01
30
40
{
@@ -36,25 +46,35 @@ POST /synonymmaps?api-version=2023-11-01
36
46
}
37
47
```
38
48
39
-
To create a synonym map, do so programmatically (the portal doesn't support synonym map definitions):
49
+
### [.NET](#tab/dotnet)
50
+
51
+
Use the [SynonymMap class (.NET)](/dotnet/api/azure.search.documents.indexes.models.synonymmap) and [Create a synonym map(Azure SDK sample)](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample02_Service.md#create-a-synonym-map) to create the map.
52
+
53
+
### [Python](#tab/python)
54
+
55
+
Use the [SynonymMap class (Python)](/python/api/azure-search-documents/azure.search.documents.indexes.models.synonymmap) to create the map.
56
+
57
+
### [Java](#tab/java)
58
+
59
+
Use the [SynonymMap class (Java)](/java/api/com.azure.search.documents.indexes.models.synonymmap) to create the map.
40
60
41
-
+[Create Synonym Map (REST API)](/rest/api/searchservice/create-synonym-map). This reference is the most descriptive.
42
-
+[SynonymMap class (.NET)](/dotnet/api/azure.search.documents.indexes.models.synonymmap) and [Create a synonym map(Azure SDK sample)](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample02_Service.md#create-a-synonym-map)
43
-
+[SynonymMap class (Python)](/python/api/azure-search-documents/azure.search.documents.indexes.models.synonymmap)
+[SynonymMap class (Java)](/java/api/com.azure.search.documents.indexes.models.synonymmap)
61
+
### [JavaScript](#tab/javascript)
46
62
47
-
## Define rules
63
+
Use the [SynonymMap interface (JavaScript)](/javascript/api/@azure/search-documents/synonymmap) to create the map.
64
+
65
+
---
66
+
67
+
### Define rules
48
68
49
69
Mapping rules adhere to the open-source synonym filter specification of Apache Solr, described in this document: [SynonymFilter](https://cwiki.apache.org/confluence/display/solr/Filter+Descriptions#FilterDescriptions-SynonymFilter). The `solr` format supports two kinds of rules:
50
70
51
-
+ equivalency (where terms are equal substitutes in the query)
71
+
- equivalency (where terms are equal substitutes in the query)
52
72
53
-
+ explicit mappings (where terms are mapped to one explicit term prior to querying)
73
+
- explicit mappings (where terms are mapped to one explicit term)
54
74
55
-
Each rule must be delimited by the new line character (`\n`). You can define up to 5,000 rules per synonym map in a free service and 20,000 rules per map in other tiers. Each rule can have up to 20 expansions (or items in a rule). For more information, see [Synonym limits](search-limits-quotas-capacity.md#synonym-limits).
75
+
Each rule is delimited by the new line character (`\n`). You can define up to 5,000 rules per synonym map in a free service and 20,000 rules per map in other tiers. Each rule can have up to 20 expansions (or items in a rule). For more information, see [Synonym limits](search-limits-quotas-capacity.md#synonym-limits).
56
76
57
-
Query parsers automatically lower-case any upper or mixed case terms, but if you want to preserve special characters in the string, such as a comma or dash, add the appropriate escape characters when creating the synonym map.
77
+
Query parsers automatically lower-case any upper or mixed case terms. To preserve special characters in the string, such as a comma or dash, add the appropriate escape characters when creating the synonym map.
58
78
59
79
### Equivalency rules
60
80
@@ -89,12 +109,12 @@ In the explicit case, a query for `Washington`, `Wash.` or `WA` is rewritten as
89
109
90
110
### Escaping special characters
91
111
92
-
In full text search, synonyms are analyzed during query processing just like any other query term, which means that rules around reserved and special characters apply to the terms in your synonym map. The list of characters that requires escaping varies between the simple syntax and full syntax:
112
+
Synonyms are analyzed during query processing just like any other query term, which means that rules for reserved and special characters apply to the terms in your synonym map. The list of characters that requires escaping varies between the simple syntax and full syntax:
Recall that if you need to preserve characters that would otherwise be discarded by the default analyzer during indexing, you should substitute an analyzer that preserves them. Some choices include Microsoft natural [language analyzers](index-add-language-analyzers.md), which preserves hyphenated words, or a custom analyzer for more complex patterns. For more information, see [Partial terms, patterns, and special characters](search-query-partial-matching.md).
117
+
To preserve characters that the default analyzer discards, substitute an analyzer that preserves them. Some choices include Microsoft natural [language analyzers](index-add-language-analyzers.md), which preserves hyphenated words, or a custom analyzer for more complex patterns. For more information, see [Partial terms, patterns, and special characters](search-query-partial-matching.md).
98
118
99
119
The following example shows an example of how to escape a character with a backslash:
100
120
@@ -105,7 +125,7 @@ The following example shows an example of how to escape a character with a backs
105
125
}
106
126
```
107
127
108
-
Since the backslash is itself a special character in other languages like JSON and C#, you'll probably need to double-escape it. For example, the JSON sent to the REST API for the above synonym map would look like this:
128
+
Since the backslash is itself a special character in other languages like JSON and C#, you probably need to double-escape it. Here's an example in JSON:
109
129
110
130
```json
111
131
{
@@ -114,15 +134,25 @@ Since the backslash is itself a special character in other languages like JSON a
114
134
}
115
135
```
116
136
117
-
## Upload and manage synonym maps
137
+
## Manage synonym maps
118
138
119
-
As mentioned previously, you can create or update a synonym map without disrupting query and indexing workloads. A synonym map is a standalone object (like indexes or data sources), and as long as no field is using it, updates won't cause indexing or queries to fail. However, once you add a synonym map to a field definition, if you then delete a synonym map, any query that includes the fields in question will fail with a 404 error.
139
+
You can update a synonym map without disrupting query and indexing workloads. However, once you add a synonym map to a field, if you then delete a synonym map, any query that includes the fields in question fail with a 404 error.
120
140
121
-
Creating, updating, and deleting a synonym map is always a whole-document operation, meaning that you can't update or delete parts of the synonym map incrementally. Updating even a single rule requires a reload.
141
+
Creating, updating, and deleting a synonym map is always a whole-document operation. You can't update or delete parts of the synonym map incrementally. Updating even a single rule requires a reload.
122
142
123
143
## Assign synonyms to fields
124
144
125
-
After uploading a synonym map, you can enable the synonyms on fields of the type `Edm.String` or `Collection(Edm.String)`, on fields having `"searchable":true`. As noted, a field definition can use only one synonym map.
145
+
After you create the synonym map, assign it to a field in your index. To assign synonym maps, do so programmatically. The portal doesn't support synonym map field associations.
146
+
147
+
- A field must be of type `Edm.String` or `Collection(Edm.String)`
148
+
- A field must have `"searchable":true`
149
+
- A field can have only one synonym map
150
+
151
+
If the synonym map exists on the search service, it's used on the next query, with no reindexing or rebuild required.
152
+
153
+
### [REST](#tab/rest-assign)
154
+
155
+
Use the [Create or Update Index (REST API)](/rest/api/searchservice/indexes/create-or-update) to modify a field definition.
126
156
127
157
```http
128
158
POST /indexes?api-version=2023-11-01
@@ -150,15 +180,63 @@ POST /indexes?api-version=2023-11-01
150
180
}
151
181
```
152
182
183
+
### [**.NET SDK**](#tab/dotnet-assign)
184
+
185
+
Use the [**SearchIndexClient**](/dotnet/api/azure.search.documents.indexes.searchindexclient) to update an index. Provide the whole index definition and include the new parameters for synonym map assignments.
186
+
187
+
In this example, the "country" field has a synonymMapName property.
For more examples, see[azure-search-dotnet-samples/quickstart/v11/](https://github.com/Azure-Samples/azure-search-dotnet-samples/tree/main/quickstart/v11).
218
+
219
+
### [**Other SDKs**](#tab/other-sdks-assign)
220
+
221
+
You can use any supported SDK to update a search index. All of them provide a **SearchIndexClient** that has methods for updating indexes.
Adding synonyms doesn't impose new requirements on query construction. You can issue term and phrase queries just as you did before the addition of synonyms. The only difference is that if a query term exists in the synonym map, the query engine will either expand or rewrite the term or phrase, depending on the rule.
233
+
A synonym field assignment doesn't change how you write queries. After the synonym map assignment, the only difference is that if a query term exists in the synonym map, the search engine either expands or rewrites the term or phrase, depending on the rule.
156
234
157
235
## How synonyms are used during query execution
158
236
159
-
Synonyms are a query expansion technique that supplements the contents of an index with equivalent terms, but only for fields that have a synonym assignment. If a field-scoped query *excludes* a synonym-enabled field, you won't see matches from the synonym map.
237
+
Synonyms are a query expansion technique that supplements the contents of an index with equivalent terms, but only for fields that have a synonym assignment. If a field-scoped query *excludes* a synonym-enabled field, you don't see matches from the synonym map.
160
238
161
-
For synonym-enabled fields, synonyms are subject to the same text analysis as the associated field. For example, if a field is analyzed using the standard Lucene analyzer, synonym terms will also be subject to the standard Lucene analyzer at query time. If you want to preserve punctuation, such as periods or dashes, in the synonym term, apply a content-preserving analyzer on the field.
239
+
For synonym-enabled fields, synonyms are subject to the same text analysis as the associated field. For example, if a field is analyzed using the standard Lucene analyzer, synonym terms are also subject to the standard Lucene analyzer at query time. If you want to preserve punctuation, such as periods or dashes, in the synonym term, apply a content-preserving analyzer on the field.
162
240
163
241
Internally, the synonyms feature rewrites the original query with synonyms with the OR operator. For this reason, hit highlighting and scoring profiles treat the original term and synonyms as equivalent.
164
242
@@ -168,7 +246,7 @@ Synonym expansions don't apply to wildcard search terms; prefix, fuzzy, and rege
168
246
169
247
If you need to do a single query that applies synonym expansion and wildcard, regex, or fuzzy searches, you can combine the queries using the OR syntax. For example, to combine synonyms with wildcards for simple query syntax, the term would be `<query> | <query>*`.
170
248
171
-
If you have an existing index in a development (non-production) environment, experiment with a small dictionary to see how the addition of synonyms changes the search experience, including impact on scoring profiles, hit highlighting, and suggestions.
249
+
If you have an existing index in a development (nonproduction) environment, experiment with a small dictionary to see how the addition of synonyms changes the search experience, including impact on scoring profiles, hit highlighting, and suggestions.
Copy file name to clipboardExpand all lines: articles/search/semantic-how-to-query-request.md
+4-8Lines changed: 4 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ ms.service: cognitive-search
10
10
ms.custom:
11
11
- ignite-2023
12
12
ms.topic: how-to
13
-
ms.date: 06/13/2024
13
+
ms.date: 07/24/2024
14
14
---
15
15
16
16
# Add semantic ranking to queries in Azure AI Search
@@ -43,11 +43,9 @@ You can use any of the following tools and SDKs to build a query that uses seman
43
43
44
44
## Avoid features that bypass relevance scoring
45
45
46
-
Several query capabilities in Azure AI Search bypass relevance scoring or are otherwise incompatible with semantic ranking. If your query logic includes the following features, you can't semantically rank your results:
46
+
A few query capabilities bypass relevance scoring, which makes them incompatible with semantic ranking. If your query logic includes the following features, you can't semantically rank your results:
47
47
48
-
+ A query with `search=*` or an empty search string, such as pure filter-only query, won't work because there's nothing to measure semantic relevance against. The query must provide terms or phrases that can be assessed during processing.
49
-
50
-
+ A query composed in the [full Lucene syntax](query-lucene-syntax.md) (`queryType=full`) is incompatible with semantic ranking (`queryType=semantic`). The semantic model doesn't support the full Lucene syntax.
48
+
+ A query with `search=*` or an empty search string, such as pure filter-only query, won't work because there's nothing to measure semantic relevance against and so the search scores are zero. The query must provide terms or phrases that can be evaluated during processing.
51
49
52
50
+ Sorting (orderBy clauses) on specific fields overrides search scores and a semantic score. Given that the semantic score is supposed to provide the ranking, adding an orderby clause results in an HTTP 400 error if you apply semantic ranking over ordered results.
53
51
@@ -116,9 +114,7 @@ The following example in this section uses the [hotels-sample-index](search-get-
116
114
117
115
1. Set "queryType" to "semantic".
118
116
119
-
In other queries, the "queryType" is used to specify the query parser. In semantic ranking, it's set to "semantic". For the "search" field, you can specify queries that conform to the [simple syntax](query-simple-syntax.md).
120
-
121
-
1. Set "search" to a full text search query based on the [simple syntax](query-simple-syntax.md). Semantic ranking is an extension of full text search, so while this parameter isn't required, you won't get an expected outcome if it's null.
117
+
1. Set "search" to a full text search query. Your search string can support either the [simple syntax](query-simple-syntax.md) or [full Lucene syntax](query-lucene-syntax.md). Semantic ranking is an extension of full text search, so while "search" isn't required, you won't get an expected outcome if it's an empty search (`"search": "*"`).
122
118
123
119
1. Set "semanticConfiguration" to a [predefined semantic configuration](semantic-how-to-configure.md) that's embedded in your index.
0 commit comments