Skip to content

Commit e319162

Browse files
committed
Edits for consistency and continuity
1 parent df0a2d3 commit e319162

File tree

2 files changed

+29
-26
lines changed

2 files changed

+29
-26
lines changed

articles/search/cognitive-search-output-field-mapping.md

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Map skill output fields
2+
title: Map enrichments in indexers
33
titleSuffix: Azure Cognitive Search
44
description: Export the enriched content created by a skillset by mapping its output fields to fields in a search index.
55

@@ -30,15 +30,15 @@ In contrast with a [`fieldMappings`](search-indexer-field-mappings.md) definitio
3030

3131
Output field mappings are required if your indexer has an attached [skillset](cognitive-search-working-with-skillsets.md) that creates new information, such as text translation or key phrase extraction. During indexer execution, AI-generated information exists in memory only. To persist this information in a search index, you'll need to tell the indexer where to send the data.
3232

33-
Output field mappings can also be used to [flatten nested data structures]() during indexing. A regular [fieldMapping definition](search-indexer-field-mappings.md) doesn't support target fields of a complex type. If you need to set up field associations for hierarchical or nested data structures, you can use a skillset and an output field mapping to create the data path.
33+
Output field mappings can also be used to retrieve specific nodes in a source document's complex type. If you don't need the full complex structure, you can [flatten individual nodes in a nested data structures](#flattening-information-from-complex-types) into a string collection.
3434

3535
Output field mappings apply to:
3636

3737
+ Content that's created by skills or extracted by an indexer. The source field is a node in an enriched document residing in memory.
3838

3939
+ Search indexes. If you're populating a [knowledge store](knowledge-store-concept-intro.md), use [projections](knowledge-store-projections-examples.md) for data path configuration.
4040

41-
Output field mappings always occur after [skillset execution](cognitive-search-working-with-skillsets.md), although it's possible to processing an output field mapping if no skillset is defined. See
41+
Output field mappings are applied after [skillset execution](cognitive-search-working-with-skillsets.md) or after document cracking if there's no associated skillset.
4242

4343
## Define an output field mapping
4444

@@ -48,7 +48,7 @@ Output field mappings are added to the `outputFieldMappings` array in an indexer
4848
"fieldMappings": []
4949
"outputFieldMappings": [
5050
{
51-
"sourceFieldName": "document/path-to-a-node-in-an-enriched-document",
51+
"sourceFieldName": "/document/path-to-a-node-in-an-enriched-document",
5252
"targetFieldName": "some-search-field-in-an-index",
5353
"mappingFunction": null
5454
}
@@ -57,14 +57,14 @@ Output field mappings are added to the `outputFieldMappings` array in an indexer
5757

5858
| Property | Description |
5959
|----------|-------------|
60-
| "sourceFieldName" | Required. Specifies a path to enriched content. See [Reference annotations in an Azure Cognitive Search skillset](cognitive-search-concept-annotations-syntax.md) for path syntax. |
61-
| "targetFieldName" | Optional. Specifies the search field that receives the enriched content. This is always a single top-level field or a collection. Target fields must be top-level simple fields or collections. It can't be a path to a subfield in a complex type. Although a target field can't resolve to subfield, you can [flatten a nested source structure into a string collection](#flatten-information-from-complex-types) in memory, and then send it to a string collection in your index. |
62-
| "mappingFunction" | Optional. Adds extra processing provided by [mapping functions](search-indexer-field-mappings.md#mappingFunctions) supported by indexers. In the case of enrichment nodes, encoding and decoding are the most commonly used functions. |
60+
| sourceFieldName | Required. Specifies a path to enriched content. An example might be `/document/content`. See [Reference annotations in an Azure Cognitive Search skillset](cognitive-search-concept-annotations-syntax.md) for path syntax and examples. |
61+
| targetFieldName | Optional. Specifies the search field that receives the enriched content. Target fields must be top-level simple fields or collections. It can't be a path to a subfield in a complex type. If you want to retrieve specific nodes in a complex structure, you can [flatten individual nodes](#flatten-information-from-complex-types) in memory, and then send the output to a string collection in your index. |
62+
| mappingFunction | Optional. Adds extra processing provided by [mapping functions](search-indexer-field-mappings.md#mappingFunctions) supported by indexers. In the case of enrichment nodes, encoding and decoding are the most commonly used functions. |
6363

6464
You can use the REST API or an Azure SDK to define output field mappings.
6565

6666
> [!TIP]
67-
> Indexers created by the [Import data wizard](search-import-data-portal.md) include output field mappings generated by the wizard. If you need examples, run the wizard over your data source to see the full definition.
67+
> Indexers created by the [Import data wizard](search-import-data-portal.md) include output field mappings generated by the wizard. If you need examples, run the wizard over your data source to see the rendered definition.
6868
6969
### [**REST APIs**](#tab/rest)
7070

@@ -102,7 +102,7 @@ api-key: [admin key]
102102
}
103103
```
104104

105-
For each output field mapping, set the location of the data in the enriched document tree (sourceFieldName), and the name of the field as referenced in the index (targetFieldName). Assign any [mapping functions](search-indexer-field-mappings.md#field-mapping-functions-and-examples) that you require to transform the content of a field before it's stored in the index.
105+
For each output field mapping, set the location of the data in the enriched document tree (sourceFieldName), and the name of the field as referenced in the index (targetFieldName). Assign any [mapping functions](search-indexer-field-mappings.md#mappingFunctions) that you require to transform the content of a field before it's stored in the index.
106106

107107
### [**.NET SDK (C#)**](#tab/csharp)
108108

@@ -132,13 +132,15 @@ await indexerClient.CreateIndexerAsync(indexer);
132132

133133
---
134134

135-
<a name="flatten-information-from-complex-types"></a>
135+
<a name="flattening-information-from-complex-types"></a>
136136

137137
## Flatten complex structures into a string collection
138138

139-
If your source data is composed of nested or hierarchical JSON, you can't use field mappings to set up the data paths. Instead, your search index must mirror the source data structure for at each level for a full import.
139+
If your source data is composed of nested or hierarchical JSON, you can't use field mappings to set up the data paths. Instead, your search index must mirror the source data structure for at each level for a full import.
140+
141+
This section walks you through an import process that produces a one-to-one reflection of a complex document on both the source and target sides. Next, it uses the same source document to illustrate the retrieval and flattening of individual nodes into string collections.
140142

141-
Sample source JSON document in Cosmos DB with nested JSON:
143+
Here's an example of a document in Cosmos DB with nested JSON:
142144

143145
```json
144146
{
@@ -171,7 +173,7 @@ Sample source JSON document in Cosmos DB with nested JSON:
171173
}
172174
```
173175

174-
Sample index definition in Cognitive Search, where names, levels, and types are reflected as a complex type:
176+
If you wanted to fully index the above source document, you'd create an index definition where the field names, levels, and types are reflected as a complex type. Because field mappings aren't supported for complex types in the search index, your index definition must mirror the source document.
175177

176178
```json
177179
{
@@ -200,7 +202,7 @@ Sample index definition in Cognitive Search, where names, levels, and types are
200202
}
201203
```
202204

203-
Sample indexer definition that runs the import (notice there are no field mappings and no skillset):
205+
Here's a sample indexer definition that executes the import (notice there are no field mappings and no skillset).
204206

205207
```json
206208
{
@@ -214,7 +216,7 @@ Sample indexer definition that runs the import (notice there are no field mappin
214216
}
215217
```
216218

217-
Sample search document, post-import, is similar to the original in Cosmos DB:
219+
The result is the following sample search document, similar to the original in Cosmos DB.
218220

219221
```json
220222
{
@@ -253,9 +255,9 @@ Sample search document, post-import, is similar to the original in Cosmos DB:
253255
}
254256
```
255257

256-
An alternative rendering in a search index is to flatten the source's nested structure to a string collection in a search index.
258+
An alternative rendering in a search index is to flatten individual nodes in the source's nested structure into a string collection in a search index.
257259

258-
To accomplish this task, you'll need an `outputFieldMapping` that maps an in-memory node to a string collection in the index. Although output field mappings primarily apply to skill outputs, you can also use them to address nodes after "document cracking" where the indexer opens a source document and reads it into memory.
260+
To accomplish this task, you'll need an `outputFieldMapping` that maps an in-memory node to a string collection in the index. Although output field mappings primarily apply to skill outputs, you can also use them to address nodes after ["document cracking"](search-indexer-overview.md#stage-1-document-cracking) where the indexer opens a source document and reads it into memory.
259261

260262
Below is a sample index definition in Cognitive Search, using string collections to receive flattened output:
261263

articles/search/search-indexer-field-mappings.md

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Field mappings in indexers
2+
title: Map fields in indexers
33
titleSuffix: Azure Cognitive Search
44
description: Configure field mappings in an indexer to account for differences in field names and data representations.
55

@@ -16,19 +16,20 @@ ms.date: 09/14/2022
1616

1717
![Indexer Stages](./media/search-indexer-field-mappings/indexer-stages-field-mappings.png "indexer stages")
1818

19-
When using an [Azure Cognitive Search indexer](search-indexer-overview.md) to push content into a search index, the indexer can often detect and assign source-to-destination field mappings. Implicit field mappings occur when field names and data types are compatible.
19+
When using an [Azure Cognitive Search indexer](search-indexer-overview.md) to pull content into a search index, the indexer can often implicitly detect and assign source-to-destination field mappings. Implicit field mappings occur when field names and data types are compatible.
2020

21-
If inputs and outputs don't match, you can define explicit *field mappings* to set up the data path, as described in this article. Field mappings can also be used to provide light-weight data conversion, such as encoding or decoding, through [mapping functions](#mappingFunctions). If more processing is required, consider [Azure Data Factory](../data-factory/index.yml) to bridge the gap.
21+
If inputs and outputs don't match, you can define explicit *field mappings* to set up the data path, as described in this article. Field mappings can also be used to introduce light-weight data conversion, such as encoding or decoding, through [mapping functions](#mappingFunctions). If more processing is required, consider [Azure Data Factory](../data-factory/index.yml) to bridge the gap.
2222

2323
Field mappings apply to:
2424

25-
+ Physical data structures on both sides of the data stream (that is, between a [supported data source](search-indexer-overview.md#supported-data-sources) and a [search index](search-what-is-an-index.md)). If you're importing enriched content created by skills, use [outputFieldMappings](cognitive-search-output-field-mapping.md) instead.
25+
+ Physical data structures on both sides of the data stream (between a [supported data source](search-indexer-overview.md#supported-data-sources) and a [search index](search-what-is-an-index.md)). If you're importing skill enriched content that resides in memory, use [outputFieldMappings](cognitive-search-output-field-mapping.md) instead.
2626

2727
+ Search indexes only. If you're populating a [knowledge store](knowledge-store-concept-intro.md), use [projections](knowledge-store-projections-examples.md) for data path configuration.
2828

2929
+ Top-level search fields only, where the "targetFieldName" is either a simple field or a collection. A target field can't be a complex type.
3030

31-
If you're working with complex data (nested or hierarchical structures), and you'd like to mirror that data structure in your search index, your search index must match the source structure exactly (same field names, levels, and types) so that the default mappings will work. Optionally, you can flatten incoming data into a string collection (see [outputFieldMappings](cognitive-search-output-field-mapping.md) for this workaround).
31+
> [!NOTE]
32+
> If you're working with complex data (nested or hierarchical structures), and you'd like to mirror that data structure in your search index, your search index must match the source structure exactly (same field names, levels, and types) so that the default mappings will work. Optionally, you might want just a few nodes in the complex structure. To get individual nodes, you can flatten incoming data into a string collection (see [outputFieldMappings](cognitive-search-output-field-mapping.md#flatten-complex-structures-into-a-string-collection) for this workaround).
3233
3334
## Supported scenarios
3435

@@ -56,9 +57,9 @@ Field mappings are added to the "fieldMappings" array of an indexer definition.
5657

5758
| Property | Description |
5859
|----------|-------------|
59-
| "sourceFieldName" | Required. Represents a field in your data source. |
60-
| "targetFieldName" | Optional. Represents a field in your search index. If omitted, the value of "sourceFieldName" is assumed for the target. Target fields must be top-level simple fields or collections. It can't be a complex type or collection. |
61-
| "mappingFunction" | Optional. Consists of [predefined functions](#mappingFunctions) that transform data. You can apply functions to both source and target field mappings. |
60+
| sourceFieldName | Required. Represents a field in your data source. |
61+
| targetFieldName | Optional. Represents a field in your search index. If omitted, the value of "sourceFieldName" is assumed for the target. Target fields must be top-level simple fields or collections. It can't be a complex type or collection. If you're handling a data type issue, a field's data type is specified in the index definition. The field mapping just needs to have the field's name.|
62+
| mappingFunction | Optional. Consists of [predefined functions](#mappingFunctions) that transform data. |
6263

6364
Azure Cognitive Search uses case-insensitive comparison to resolve the field and function names in field mappings. This is convenient (you don't have to get all the casing right), but it means that your data source or index can't have fields that differ only by case.
6465

@@ -84,7 +85,7 @@ api-key: [admin key]
8485
}
8586
```
8687

87-
Map a single source field to multiple target fields ("one-to-many" mappings). The following example shows how to "fork" a field, copying the same source field content to two different index fields that will be analyzed or attributed differently in the index.
88+
This example maps a single source field to multiple target fields ("one-to-many" mappings). You can "fork" a field, copying the same source field content to two different index fields that will be analyzed or attributed differently in the index.
8889

8990
```JSON
9091

0 commit comments

Comments
 (0)