Skip to content

Commit ec610b5

Browse files
authored
Merge pull request #89415 from vkurpad/master
removed cache references
2 parents d858392 + 2ea0ed3 commit ec610b5

File tree

1 file changed

+16
-11
lines changed

1 file changed

+16
-11
lines changed

articles/search/knowledge-store-concept-intro.md

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,13 @@ Knowledge store is a feature of Azure Search that saves enriched documents and m
2121

2222
If you have used cognitive search in the past, you already know that skillsets are used to move a document through a sequence of enrichments. The outcome can be an Azure Search index, or (new in this preview) projections in a knowledge store. The two outputs, search index and knowledge store, are physically distinct from each other. They share the same content, but are stored and used in very different ways.
2323

24-
Physically, a knowledge store is created in an Azure Storage account, either as Azure Table storage or Blob storage, depending on how you configure the pipeline. Any tool or process that can connect to Azure Storage can consume the contents of a knowledge store.
24+
Physically, a knowledge store is an Azure Storage account, either as Azure Table storage, Blob storage or both, depending on how you configure the pipeline. Any tool or process that can connect to Azure Storage can consume the contents of a knowledge store.
2525

2626
Projections are your mechanism for structuring data in a knowledge store. For example, through projections, you can choose whether output is saved as a single blob or a collection of related tables. An easy way to view knowledge store contents is through the built-in [Storage Explorer](https://docs.microsoft.com/azure/vs-azure-tools-storage-manage-with-storage-explorer?tabs=windows) for Azure storage.
2727

2828
![Knowledge store in pipeline diagram](./media/knowledge-store-concept-intro/annotationstore_sans_internalcache.png "Knowledge store in pipeline diagram")
2929

30-
To use knowledge store, add a `knowledgeStore` element to a skillset that defines step-wise operations in an indexing pipeline. During execution, Azure Search creates a space in your Azure storage account and fills it with definitions and content created by the pipeline.
30+
To use knowledge store, add a `knowledgeStore` element to a skillset that defines step-wise operations in an indexing pipeline. During execution, Azure Search creates a space in your Azure storage account and projects the enriched documents with the definition created within the pipeline.
3131

3232
## Benefits of knowledge store
3333

@@ -98,6 +98,13 @@ If you are already familiar with AI-based indexing, the skillset definition dete
9898
{ "tableName": "KeyPhrases", "generatedKeyName": "KeyPhraseId", "source": "/document/Review/Sentences/*/KeyPhrases", "sourceContext": null, "inputs": []},
9999
{ "tableName": "Entities", "generatedKeyName": "EntityId", "source": "/document/Review/Sentences/*/Entities/*" ,"sourceContext": null, "inputs": []}
100100

101+
],
102+
"objects": [
103+
104+
]
105+
},
106+
{
107+
"tables": [
101108
],
102109
"objects": [
103110
{
@@ -107,7 +114,7 @@ If you are already familiar with AI-based indexing, the skillset definition dete
107114
"key": "/document/Review/Id"
108115
}
109116
]
110-
}
117+
}
111118
]
112119
}
113120
}
@@ -138,17 +145,17 @@ Azure Search provides the indexer feature, and indexers are used to drive the en
138145
| Object | REST API | Description |
139146
|--------|----------|-------------|
140147
| data source | [Create Data Source](https://docs.microsoft.com/rest/api/searchservice/create-data-source) | A resource identifying an external Azure data source providing source data used to create enriched documents. |
141-
| skillset | [Create Skillset (api-version=2019-05-06)](https://docs.microsoft.com/rest/api/searchservice/create-skillset) | A resource coordinating the use of [built-in skills](cognitive-search-predefined-skills.md) and [custom cognitive skills](cognitive-search-custom-skill-interface.md) used in an enrichment pipeline during indexing. |
148+
| skillset | [Create Skillset (api-version=2019-05-06-Preview)](https://docs.microsoft.com/rest/api/searchservice/create-skillset) | A resource coordinating the use of [built-in skills](cognitive-search-predefined-skills.md) and [custom cognitive skills](cognitive-search-custom-skill-interface.md) used in an enrichment pipeline during indexing. |
142149
| index | [Create Index](https://docs.microsoft.com/rest/api/searchservice/create-index) | A schema expressing an Azure Search index. Fields in the index map to fields in source data or to fields manufactured during the enrichment phase (for example, a field for organization names created by entity recognition). |
143150
| indexer | [Create Indexer (api-version=2019-05-06)](https://docs.microsoft.com/rest/api/searchservice/create-skillset) | A resource defining components used during indexing: including a data source, a skillset, field associations from source and intermediary data structures to target index, and the index itself. Running the indexer is the trigger for data ingestion and enrichment. The output is a search index based on the index schema, populated with source data, enriched through skillsets. |
144151

145152
### 3 - Cognitive Services
146153

147-
Enrichments specified in a skillset are based on the Computer Vision and Language features in Cognitive Services. Cognitive Services functionality is leveraged during indexing through your skillset. A skillset is a composition of skills, and skills are bound to specific Computer Vision and Language features. To integrate Cognitive Services, you'll [attach a Cognitive Services resource](cognitive-search-attach-cognitive-services.md) to a skillset.
154+
Enrichments specified in a skillset are either custom or based on the Computer Vision and Language features in Cognitive Services. Cognitive Services functionality is leveraged during indexing through your skillset. A skillset is a composition of skills, and skills are bound to specific Computer Vision and Language features. To integrate Cognitive Services, you'll [attach a Cognitive Services resource](cognitive-search-attach-cognitive-services.md) to a skillset.
148155

149156
### 4 - Storage account
150157

151-
Under your Azure Storage account, Azure Search creates a Blob container or tables, depending on how you configure a skillset. If your data originates from Azure Blob or Table storage, you are already set. Otherwise, you will need to create an Azure storage account. Tables and objects in Azure storage contain the enriched documents created by the AI-based indexing pipeline.
158+
Under your Azure Storage account, Azure Search creates a Blob container or tables or both, depending on how you configure projections within the skillset. If your data originates from Azure Blob or Table storage, you are already set and can reuse the storage account. Otherwise, you will need to create an Azure storage account. Tables and objects in Azure storage contain the enriched documents created by the AI-based indexing pipeline.
152159

153160
The storage account is specified in the skillset. In `api-version=2019-05-06-Preview`, a skillset definition includes a knowledge store definition so that you can provide account information.
154161

@@ -174,15 +181,13 @@ Within the storage account, the enrichments can be expressed as tables within Az
174181

175182
+ Blob storage creates one all-inclusive JSON representation of each document. You can use both storage options in one skillset to get a full range of expressions.
176183

177-
+ Azure Search persists content in an index. If your scenario is non-search-related, for example if your objective is analysis in another tool, you can delete the index that the pipeline creates. But you could also keep the index and use a built-in tool like [Search Explorer](search-explorer.md) as a third medium (behind Storage Explorer and your analytics app) for interacting with your content.
178-
179-
Along with document contents, enriched documents include the metadata of the skillset version that produced the enrichments.
184+
+ Azure Search persists content in an index. If your scenario is non-search-related, for example if your objective is analysis in another tool, you can delete the index that the pipeline creates. But you could also keep the index and use a built-in tool like [Search Explorer](search-explorer.md) as a third medium (behind Storage Explorer and your analytics app) for interacting with your content.
180185

181186
## Inside a knowledge store
182187

183-
The knowledge store consists of an annotation cache and projections. The *cache* is used by the service internally to cache the results from skills and track changes. A *projection* defines the schema and structure of the enrichments that match your intended use. There is one cache per knowledge store, but multiple projections.
188+
A *projection* defines the schema and structure of the enrichments that match your intended use. You can define multiple projections if you have applications that consume the data in different formats and shapes.
184189

185-
The cache is always a blob container, but projections can be articulated as tables or objects:
190+
Projections can be articulated as objects or tables:
186191

187192
+ As an object, the projection maps to Blob storage, where the projection is saved to a container, within which are the objects or hierarchical representations in JSON for scenarios like a data science pipeline.
188193

0 commit comments

Comments
 (0)