You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/cognitive-search-output-field-mapping.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,9 +26,9 @@ This article explains how to set up *output field mappings* that determine a dat
26
26
],
27
27
```
28
28
29
-
In contrast with a ["fieldMappings" definition](search-indexer-field-mappings.md) that maps a path between two physical data structures, an "outputFieldMappings" definition maps in-memory data to fields in a search index.
29
+
In contrast with a [`fieldMappings`](search-indexer-field-mappings.md)definition that maps a path between two physical data structures, an `outputFieldMappings` definition maps in-memory data to fields in a search index.
30
30
31
-
Output field mappings are required if your indexer has an attached [skillset](cognitive-search-working-with-skillsets.md) that creates new information, such as translated strings or key phrases. During indexer execution, AI-generated information exists in memory. To persist this information in a search index, you'll need to tell the indexer where to send the data.
31
+
Output field mappings are required if your indexer has an attached [skillset](cognitive-search-working-with-skillsets.md) that creates new information, such as text translation or key phrase extraction. During indexer execution, AI-generated information exists in memory only. To persist this information in a search index, you'll need to tell the indexer where to send the data.
32
32
33
33
Output field mappings can also be used to flatten nested data structures during indexing. A regular [fieldMapping definition](search-indexer-field-mappings.md) doesn't support target fields of a complex type. If you need to set up field associations for hierarchical or nested data structures, you can use a skillset and an output field mapping to create the data path.
34
34
@@ -38,7 +38,7 @@ Output field mappings apply to:
38
38
39
39
+ Search indexes. If you're populating a [knowledge store](knowledge-store-concept-intro.md), use [projections](knowledge-store-projections-examples.md) for data path configuration.
40
40
41
-
Output field mappings always occur after [skillset execution](cognitive-search-working-with-skillsets.md), although it is possible for this stage to run even if no skillset is defined.
41
+
Output field mappings always occur after [skillset execution](cognitive-search-working-with-skillsets.md), although it's possible for this stage to run even if no skillset is defined.
42
42
43
43
<!--
44
44
The enriched document is really a tree of information, and even though there is support for complex types in the index, sometimes you may want to transform the information from the enriched tree into a more simple type (for instance, an array of strings).
@@ -51,9 +51,9 @@ Examples of output field mapping scenarios:
51
51
52
52
* **Content extraction.** You don’t have a skillset but are indexing a complex type from a Cosmos DB database. You'd like to get to a node on that complex type and map it into a field in your index. -->
53
53
54
-
## Set up an `outputFieldMappings` array
54
+
## Define an output field mapping
55
55
56
-
Output field mappings are added to the "outputFieldMappings" array in an indexer definition, typically placed after the "filedMappings" array. An output field mapping consists of three parts.
56
+
Output field mappings are added to the `outputFieldMappings` array in an indexer definition, typically placed after the `fieldMappings` array. An output field mapping consists of three parts.
57
57
58
58
```json
59
59
"fieldMappings": []
@@ -68,8 +68,8 @@ Output field mappings are added to the "outputFieldMappings" array in an indexer
68
68
69
69
| Property | Description |
70
70
|----------|-------------|
71
-
| "sourceFieldName" | Required. Specifies a path to enriched content |
72
-
| "targetFieldName" | Optional. Specifies the search field that receives the enriched content|
71
+
| "sourceFieldName" | Required. Specifies a path to enriched content. See [Reference annotations in an Azure Cognitive Search skillset](cognitive-search-concept-annotations-syntax.md) for path syntax.|
72
+
| "targetFieldName" | Optional. Specifies the search field that receives the enriched content. This is always a single top-level field or a collection. |
73
73
| "mappingFunction" | Optional. Adds extra processing provided by [search-indexer-field-mappings.md#predefined functions](#mappingFunctions) supported by indexers. In the case of enrichment nodes, encoding and decoding are the most commonly used functions. |
74
74
75
75
You can use the REST API or an Azure SDK to define output field mappings.
@@ -157,7 +157,7 @@ More concretely, for the ```/document/content/organizations/*/description``` exa
157
157
["Microsoft is a company in Seattle","LinkedIn's office is in San Francisco"]
158
158
```
159
159
160
-
This is an important principle, so we will provide another example. Imagine that you have an array of complex types as part of the enrichment tree. Let's say there is a member called customEntities that has an array of complex types like the one described below.
160
+
This is an important principle, so we'll provide another example. Imagine that you have an array of complex types as part of the enrichment tree. Let's say there's a member called customEntities that has an array of complex types like the one described below.
Copy file name to clipboardExpand all lines: articles/search/search-indexer-field-mappings.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,23 +22,23 @@ If inputs and outputs don't match, you can define explicit *field mappings* to s
22
22
23
23
Field mappings apply to:
24
24
25
-
+ Physical data structures on both sides of the data stream (that is, between a [supported data source](search-indexer-overview.md#supported-data-sources) and a search index). If you're importing enriched content created by skills, use [outputFieldMappings](cognitive-search-output-field-mapping.md) instead.
25
+
+ Physical data structures on both sides of the data stream (that is, between a [supported data source](search-indexer-overview.md#supported-data-sources) and a [search index](search-what-is-an-index.md)). If you're importing enriched content created by skills, use [outputFieldMappings](cognitive-search-output-field-mapping.md) instead.
26
26
27
-
+ Search indexes. If you're populating a [knowledge store](knowledge-store-concept-intro.md), use [projections](knowledge-store-projections-examples.md) for data path configuration.
27
+
+ Search indexes only. If you're populating a [knowledge store](knowledge-store-concept-intro.md), use [projections](knowledge-store-projections-examples.md) for data path configuration.
28
28
29
-
+ Top-level search fields only, where the "targetFieldName" is either a simple field or a collection. If you need to establish a data path to subfields in a complex type (such as mapping to `address/city`), see [this workaround using outputFieldMappings](cognitive-search-output-field-mapping.md).
29
+
+ Top-level search fields only, where the "targetFieldName" is either a simple field or a collection. If you're working with complex data (nested or hierarchical structures), see [outputFieldMappings](cognitive-search-output-field-mapping.md) for workarounds.
30
30
31
31
## Supported scenarios
32
32
33
33
| Use-case | Description |
34
34
|----------|-------------|
35
-
| Name discrepancy | Suppose your data source has a field named `_city`. Given that Azure Cognitive Search doesn't allow field names that start with an underscore, a field mapping lets you effectively map "_city" to "city". </p>If your indexing requirements includes retrieving content from multiple data sources, where field names vary among the sources,you could use a field mapping to clarify the path.|
35
+
| Name discrepancy | Suppose your data source has a field named `_city`. Given that Azure Cognitive Search doesn't allow field names that start with an underscore, a field mapping lets you effectively map "_city" to "city". </p>If your indexing requirements include retrieving content from multiple data sources, where field names vary among the sources,you could use a field mapping to clarify the path.|
36
36
| Type discrepancy | Supposed you want a source integer field to be of type `Edm.String` so that it's searchable in the search index. Because the types are different, you'll need to define a field mapping in order for the data path to succeed. Note that Cognitive Search has a smaller set of [supported data types](/rest/api/searchservice/supported-data-types) than many data sources. If you're importing SQL data, a field mapping allows you to [map the SQL data type](search-howto-connecting-azure-sql-database-to-azure-search-using-indexers.md#mapping-data-types) you want in a search index.|
37
37
| One-to-many data paths | You can populate multiple fields in the index with content from the same source field. For example, you might want to apply different analyzers to each field to support different use cases in your client app.|
38
38
| Encoding and decoding | You can apply [mapping functions](#mappingFunctions) to support Base64 encoding or decoding of data during indexing. |
39
39
| Split strings or recast arrays into collections | You can apply [mapping functions](#mappingFunctions) to split a string that includes a delimiter, or to send a JSON array to a search field of type `Collection(Edm.String)`.
40
40
41
-
## Set up a `fieldMappings` array
41
+
## Define a field mapping
42
42
43
43
Field mappings are added to the "fieldMappings" array of an indexer definition. A field mapping consists of three parts.
44
44
@@ -163,7 +163,7 @@ A document key (both before and after conversion) can't be longer than 1,024 cha
163
163
164
164
#### Example: Make a base-encoded field "searchable"
165
165
166
-
There are times when you need to use an encoded version of a field like "metadata_storage_path" as the key, but also need an un-encoded version for full text search. To support both scenarios, you can map "metadata_storage_path" to two fields: one for the key (encoded), and a second for a path field that we can assume is attributed as "searchable" in the index schema.
166
+
There are times when you need to use an encoded version of a field like "metadata_storage_path" as the key, but also need an unencoded version for full text search. To support both scenarios, you can map "metadata_storage_path" to two fields: one for the key (encoded), and a second for a path field that we can assume is attributed as "searchable" in the index schema.
167
167
168
168
```http
169
169
PUT /indexers/blob-indexer?api-version=2020-06-30
@@ -226,7 +226,7 @@ Your source data might contain Base64-encoded strings, such as blob metadata str
226
226
227
227
If you don't include a parameters property, it defaults to the value `{"useHttpServerUtilityUrlTokenEncode" : true}`.
228
228
229
-
Azure Cognitive Search supports two different Base64 encodings. You should use the same parameters when encoding and decoding the same field. For more details, see [base64 encoding options](#base64details) to decide which parameters to use.
229
+
Azure Cognitive Search supports two different Base64 encodings. You should use the same parameters when encoding and decoding the same field. For more information, see [base64 encoding options](#base64details) to decide which parameters to use.
230
230
231
231
<aname="base64details"></a>
232
232
@@ -239,11 +239,11 @@ If the `useHttpServerUtilityUrlTokenEncode` or `useHttpServerUtilityUrlTokenDeco
239
239
> [!WARNING]
240
240
> If `base64Encode` is used to produce key values, `useHttpServerUtilityUrlTokenEncode` must be set to true. Only URL-safe base64 encoding can be used for key values. See [Naming rules](/rest/api/searchservice/naming-rules) for the full set of restrictions on characters in key values.
241
241
242
-
The .NET libraries in Azure Cognitive Search assume the full .NET Framework, which provides built-in encoding. The `useHttpServerUtilityUrlTokenEncode` and `useHttpServerUtilityUrlTokenDecode` options leverage this built-in functionality. If you're using .NET Core or another framework, we recommend setting those options to `false` and calling your framework's encoding and decoding functions directly.
242
+
The .NET libraries in Azure Cognitive Search assume the full .NET Framework, which provides built-in encoding. The `useHttpServerUtilityUrlTokenEncode` and `useHttpServerUtilityUrlTokenDecode` options apply this built-in functionality. If you're using .NET Core or another framework, we recommend setting those options to `false` and calling your framework's encoding and decoding functions directly.
243
243
244
244
The following table compares different base64 encodings of the string `00>00?00`. To determine the required processing (if any) for your base64 functions, apply your library encode function on the string `00>00?00` and compare the output with the expected output `MDA-MDA_MDA`.
245
245
246
-
| Encoding | Base64 encode output |Additional processing after library encoding |Additional processing before library decoding |
246
+
| Encoding | Base64 encode output |Extra processing after library encoding |Extra processing before library decoding |
247
247
| --- | --- | --- | --- |
248
248
| Base64 with padding |`MDA+MDA/MDA=`| Use URL-safe characters and remove padding | Use standard base64 characters and add padding |
249
249
| Base64 without padding |`MDA+MDA/MDA`| Use URL-safe characters | Use standard base64 characters |
0 commit comments