Merge pull request #99878 from HeidiSteen/heidist-gh-kstore

Azure Cognitive Search: kstore walkthrough revs

Author: ktoliver

articles/search/knowledge-store-concept-intro.md

@@ -17,9 +17,9 @@ ms.date: 12/11/2019
 > Knowledge store is currently in public preview. Preview functionality is provided without a service level agreement, and is not recommended for production workloads. For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). 
 > The [REST API version 2019-05-06-Preview](search-api-preview.md) provides preview features. There is currently limited portal support, and no .NET SDK support.
 
-A knowledge store is a feature of Azure Cognitive Search that persists output from an [AI enrichment pipeline](cognitive-search-concept-intro.md) for independent analysis or downstream processing. An *enriched document* is a pipeline's output, created from content that has been extracted, structured, and analyzed using AI processes. In a standard AI pipeline, enriched documents are transitory, used only during indexing and then discarded. With knowledge store, enriched documents are preserved. 
+Knowledge store is a feature of Azure Cognitive Search that persists output from an [AI enrichment pipeline](cognitive-search-concept-intro.md) for independent analysis or downstream processing. An *enriched document* is a pipeline's output, created from content that has been extracted, structured, and analyzed using AI processes. In a standard AI pipeline, enriched documents are transitory, used only during indexing and then discarded. With knowledge store, enriched documents are preserved. 
 
-If you have used cognitive skills in the past, you already know that *skillsets* move a document through a sequence of enrichments. The outcome can be a search index, or (new in this preview) projections in a knowledge store. The two outputs, search index and knowledge store, derive from the same content input, but output is structured, stored, and used in very different ways.
+If you have used cognitive skills in the past, you already know that *skillsets* move a document through a sequence of enrichments. The outcome can be a search index, or (new in this preview) projections in a knowledge store. The two outputs, search index and knowledge store, are products of the same pipeline; derived from the same inputs, but resulting in output that is structured, stored, and used in very different ways.
 
 Physically, a knowledge store is [Azure Storage](https://docs.microsoft.com/azure/storage/common/storage-account-overview), either Azure Table storage, Azure Blob storage, or both. Any tool or process that can connect to Azure Storage can consume the contents of a knowledge store.
 
@@ -90,7 +90,7 @@ However, it is possible to create multiple sets of `table`-`object`-`file` proje
 
 + Indexers have schedules and other properties, such as change detection mechanisms provided by various data sources, can also be applied to a knowledge store. For example, you can [schedule](search-howto-schedule-indexers.md) enrichment at regular intervals to refresh the contents. 
 
-## Create a knowledge store
+## How to create a knowledge store
 
 To create knowledge store, use the portal or the preview REST API (`api-version=2019-05-06-Preview`).
 
@@ -116,13 +116,13 @@ Reference content for this preview feature is located in the [API reference](#ks
 
 <a name="tools-and-apps"></a>
 
-## Connect with tools and apps
+## How to connect with tools and apps
 
 Once the enrichments exist in storage, any tool or technology that connects to Azure Blob or Table storage can be used to explore, analyze, or consume the contents. The following list is a start:
 
 + [Storage Explorer](knowledge-store-view-storage-explorer.md) to view enriched document structure and content. Consider this as your baseline tool for viewing knowledge store contents.
 
-+ [Power BI](knowledge-store-connect-power-bi.md) for the reporting and analysis tools if you have numeric data.
++ [Power BI](knowledge-store-connect-power-bi.md) for reporting and analysis. Visualizations are easier if enriched documents contain numeric data. For text, you can build word clouds, use custom visualizations, or consider [Q&A](https://docs.microsoft.com/power-bi/consumer/end-user-q-and-a) for natural language exploration.
 
 + [Azure Data Factory](https://docs.microsoft.com/azure/data-factory/) for further manipulation.
 

articles/search/knowledge-store-create-portal.md

@@ -8,7 +8,7 @@ ms.author: heidist
 manager: nitinme
 ms.service: cognitive-search
 ms.topic: quickstart
-ms.date: 11/25/2019
+ms.date: 12/30/2019
 ---
 
 # Quickstart: Create an Azure Cognitive Search knowledge store in the Azure portal
@@ -18,7 +18,7 @@ ms.date: 11/25/2019
 
 Knowledge store is a feature of Azure Cognitive Search that persists output from a cognitive skills pipeline for subsequent analyses or downstream processing. 
 
-A pipeline accepts images and unstructured text as raw content, applies AI through Cognitive Services (such as image and natural language processing), extracts information, and outputs new structures and information. One of the physical artifacts created by a pipeline is a [knowledge store](knowledge-store-concept-intro.md), which you can access through tools to analyze and explore content.
+A pipeline accepts unstructured text and images as raw content, applies AI through Cognitive Services (such as OCR, image analysis, and natural language processing), extracts information, and outputs new structures and information. One of the physical artifacts created by a pipeline is a [knowledge store](knowledge-store-concept-intro.md), which you can access through tools to analyze and explore content.
 
 In this quickstart, you'll combine services and data in the Azure cloud to create a knowledge store. Once everything is in place, you'll run the **Import data** wizard in the portal to pull it all together. The end result is original text content plus AI-generated content that you can view in the portal ([Storage explorer](knowledge-store-view-storage-explorer.md)).
 
@@ -34,13 +34,9 @@ Because the workload is so small, Cognitive Services is tapped behind the scenes
 
 1. [Create an Azure storage account](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-portal) or [find an existing account](https://ms.portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Storage%2storageAccounts/) under your current subscription. You'll use Azure storage for both the raw content to be imported, and the knowledge store that is the end result.
 
-   There are two requirements for this account:
+   Choose the **StorageV2 (general purpose V2)** account type.
 
-   + Choose the same region as Azure Cognitive Search. 
-   
-   + Choose the StorageV2 (general purpose V2) account type. 
-
-1. Open the Blob services pages and create a container.  
+1. Open the Blob services pages and create a container named *hotel-reviews*.
 
 1. Click **Upload**.
 
@@ -78,18 +74,17 @@ In the search service Overview page, click **Import data** on the command bar to
 
 ### Step 2: Add cognitive skills
 
-In this wizard step, you will create a skillset with cognitive skill enrichments. The skills we use in this sample will extract key phrases and detect the language and sentiment. In a later step, these enrichments will be "projected" into a knowledge store as Azure tables.
+In this wizard step, you will create a skillset with cognitive skill enrichments. The source data consists of customer reviews in several languages. Skills that are relevant for this data set include key phrase extraction, sentiment detection, and text translation. In a later step, these enrichments will be "projected" into a knowledge store as Azure tables.
 
 1. Expand **Attach Cognitive Services**. **Free (Limited enrichments)** is selected by default. You can use this resource because number of records in HotelReviews-Free.csv is 19 and this free resource allows up to 20 transactions a day.
 1. Expand **Add cognitive skills**.
 1. For **Skillset name**, enter `hotel-reviews-ss`.
 1. For **Source data field**, select **reviews_text**.
 1. For **Enrichment granularity level**, select **Pages (5000 characters chunks)**
 1. Select these cognitive skills:
-    + **Extract people names**
-    + **Extract organization names**
-    + **Extract location names**
     + **Extract key phrases**
+    + **Translate text**
+    + **Detect sentiment**
 
       ![Create a skillset](media/knowledge-store-create-portal/hotel-reviews-ss.png "Create a skillset")
 
@@ -102,17 +97,16 @@ In this wizard step, you will create a skillset with cognitive skill enrichments
 
     ![Configure knowledge store](media/knowledge-store-create-portal/hotel-reviews-ks.png "Configure knowledge store")
 
+1. Optionally, download a Power BI template. When you access the template from the wizard, the local .pbit file is adapted to reflect the shape of your data.
+
 1. Continue to the next page.
 
 ### Step 3: Configure the index
 
 In this wizard step, you will configure an index for optional full-text search queries. The wizard will sample your data source to infer fields and data types. You only need to select the attributes for your desired behavior. For example, the **Retrievable** attribute will allow the search service to return a field value while the **Searchable** will enable full text search on the field.
 
 1. For **Index name**, enter `hotel-reviews-idx`.
-1. For attributes, make these selections:
-    + Select **Retrievable** for all fields.
-    + Select **Filterable** and **Facetable** for these fields: *Sentiment*, *Language*, *Keyphrases*
-    + Select **Searchable** for these fields: *city*, *name*, *reviews_text*, *language*, *Keyphrases*
+1. For attributes, accept the default selections: **Retrievable** and **Searchable** for the new fields that the pipeline is creating.
 
     Your index should look similar to the following image. Because the list is long, not all fields are visible in the image.
 

articles/search/knowledge-store-create-rest.md

@@ -8,7 +8,7 @@ manager: nitinme
 ms.author: heidist
 ms.service: cognitive-search
 ms.topic: tutorial
-ms.date: 11/04/2019
+ms.date: 12/30/2019
 ---
 # Create an Azure Cognitive Search knowledge store by using REST
 
@@ -22,35 +22,36 @@ In this article, you use the REST API interface to ingest, index, and apply AI e
 
 After you create the knowledge store, you can learn about how to access the knowledge store by using [Storage Explorer](knowledge-store-view-storage-explorer.md) or [Power BI](knowledge-store-connect-power-bi.md).
 
-## Create services
+If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
 
-Create the following services:
+> [!TIP]
+> We recommend [Postman desktop app](https://www.getpostman.com/) for this article. The [source code](https://github.com/Azure-Samples/azure-search-postman-samples/tree/master/knowledge-store) for this article includes a Postman collection containing all of the requests. 
 
-- Create an [Azure Cognitive Search service](search-create-service-portal.md) or [find an existing service](https://ms.portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices) in your current subscription. You can use a free service for this tutorial.
+## Create services and load data
 
-- Create an [Azure storage account](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account) to store the sample data and the knowledge store. Your storage account must use the same location (such as US-West) for your Azure Cognitive Search service. The value for **Account kind** must be **StorageV2 (general purpose V2)** (default) or **Storage (general purpose V1)**.
+This quickstart uses Azure Cognitive Search, Azure Blob storage, and [Azure Cognitive Services](https://azure.microsoft.com/services/cognitive-services/) for the AI. 
 
-- Recommended: Get the [Postman desktop app](https://www.getpostman.com/) for sending requests to Azure Cognitive Search. You can use the REST API with any tool that's capable of working with HTTP requests and responses. Postman is a good choice for exploring REST APIs. We use Postman in this article. Also, the [source code](https://github.com/Azure-Samples/azure-search-postman-samples/tree/master/knowledge-store) for this article includes a Postman collection of requests. 
+Because the workload is so small, Cognitive Services is tapped behind the scenes to provide free processing for up to 20 transactions daily when invoked from Azure Cognitive Search. As long as you use the sample data we provide, you can skip creating or attaching a Cognitive Services resource.
 
-## Store the data
+1. [Download HotelReviews_Free.csv](https://knowledgestoredemo.blob.core.windows.net/hotel-reviews/HotelReviews_Free.csv?sp=r&st=2019-11-04T01:23:53Z&se=2025-11-04T16:00:00Z&spr=https&sv=2019-02-02&sr=b&sig=siQgWOnI%2FDamhwOgxmj11qwBqqtKMaztQKFNqWx00AY%3D). This data is hotel review data saved in a CSV file (originates from Kaggle.com) and contains 19 pieces of customer feedback about a single hotel. 
 
-Load the hotel reviews CSV file into Azure Blob storage so it can be accessed by an Azure Cognitive Search indexer and fed through the AI enrichment pipeline.
+1. [Create an Azure storage account](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?tabs=azure-portal) or [find an existing account](https://ms.portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Storage%2storageAccounts/) under your current subscription. You'll use Azure storage for both the raw content to be imported, and the knowledge store that is the end result.
 
-### Create a blob container by using the data
+   Choose the **StorageV2 (general purpose V2)** account type.
 
-1. Download the [hotel review data](https://knowledgestoredemo.blob.core.windows.net/hotel-reviews/HotelReviews_Free.csv?st=2019-07-29T17%3A51%3A30Z&se=2021-07-30T17%3A51%3A00Z&sp=rl&sv=2018-03-28&sr=c&sig=LnWLXqFkPNeuuMgnohiz3jfW4ijePeT5m2SiQDdwDaQ%3D) saved in a CSV file (HotelReviews_Free.csv). This data originates from Kaggle.com and contains customer feedback about hotels.
-1. Sign in to the [Azure portal](https://portal.azure.com) and go to your Azure storage account.
-1. Create a [blob container](https://docs.microsoft.com/azure/storage/blobs/storage-quickstart-blobs-portal). To create the container, in the left menu for your storage account, select **Blobs**, and then select **Container**.
-1. For the new container **Name**, enter **hotel-reviews**.
-1. For **Public Access Level**, select any value. We used the default.
-1. Select **OK** to create the blob container.
-1. Open the new **hotels-review** container, select **Upload**, and then select the HotelReviews-Free.csv file you downloaded in the first step.
+1. Open the Blob services pages and create a container named *hotel-reviews*.
+
+1. Click **Upload**.
 
     ![Upload the data](media/knowledge-store-create-portal/upload-command-bar.png "Upload the hotel reviews")
 
-1. Select **Upload** to import the CSV file into Azure Blob storage. The new container is shown:
+1. Select the **HotelReviews-Free.csv** file you downloaded in the first step.
+
+    ![Create the Azure Blob container](media/knowledge-store-create-portal/hotel-reviews-blob-container.png "Create the Azure Blob container")
+
+1. You are almost done with this resource, but before you leave these pages, use a link on the left navigation pane to open the **Access Keys** page. Get a connection string to retrieve data from Blob storage. A connection string looks similar to the following example: `DefaultEndpointsProtocol=https;AccountName=<YOUR-ACCOUNT-NAME>;AccountKey=<YOUR-ACCOUNT-KEY>;EndpointSuffix=core.windows.net`
 
-    ![Create the blob container](media/knowledge-store-create-portal/hotel-reviews-blob-container.png "Create the blob container")
+1. Still in the portal, switch to Azure Cognitive Search. [Create a new service](search-create-service-portal.md) or [find an existing service](https://ms.portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices). You can use a free service for this exercise.
 
 ## Configure Postman
 
@@ -68,7 +69,7 @@ Install and set up Postman.
 
 On the **Variables** tab, you can add values that Postman swaps in every time it encounters a specific variable inside double braces. For example, Postman replaces the symbol `{{admin-key}}` with the current value that you set for `admin-key`. Postman makes the substitution in URLs, headers, the request body, and so on. 
 
-To get the value for `admin-key`, go to the Azure Cognitive Search service and select the **Keys** tab. Change `search-service-name` and `storage-account-name` to the values you chose in [Create services](#create-services). Set `storage-connection-string` by using the value on the storage account's **Access Keys** tab. You can leave the defaults for the other values.
+To get the value for `admin-key`, go to the Azure Cognitive Search service and select the **Keys** tab. Change `search-service-name` and `storage-account-name` to the values you chose in [Create services](#create-services-and-load-data). Set `storage-connection-string` by using the value on the storage account's **Access Keys** tab. You can leave the defaults for the other values.
 
 ![Postman app variables tab](media/knowledge-store-create-rest/postman-variables-window.png "Postman's variables window")
 
@@ -149,7 +150,7 @@ Select **Send** to issue the PUT request. You should see the status `201 - Creat
 
 ## Create the datasource
 
-Next, connect Azure Cognitive Search to the hotel data you stored in [Store the data](#store-the-data). To create the datasource, send a POST request to `https://{{search-service-name}}.search.windows.net/datasources?api-version={{api-version}}`. You must set the `api-key` and `Content-Type` headers as discussed earlier. 
+Next, connect Azure Cognitive Search to the hotel data you stored in Blob storage. To create the datasource, send a POST request to `https://{{search-service-name}}.search.windows.net/datasources?api-version={{api-version}}`. You must set the `api-key` and `Content-Type` headers as discussed earlier. 
 
 In Postman, go to the **Create Datasource** request, and then to the **Body** pane. You should see the following code:
 

articles/search/knowledge-store-view-storage-explorer.md

@@ -8,7 +8,7 @@ author: HeidiSteen
 ms.author: heidist
 ms.service: cognitive-search
 ms.topic: conceptual
-ms.date: 11/26/2019
+ms.date: 12/30/2019
 ---
 
 # View a knowledge store with Storage Explorer
@@ -21,7 +21,7 @@ In this article, you'll learn by example how to connect to and explore a knowled
 
 ## Prerequisites
 
-+ Follow the steps in [Create a knowledge store in Azure portal](knowledge-store-create-portal.md) or [Create an Azure Cognitive Search knowledge store by using REST](knowledge-store-create-rest.md) to create the sample knowledge store used in this walkthrough.
++ Follow the steps in [Create a knowledge store in Azure portal](knowledge-store-create-portal.md) to create the sample knowledge store used in this walkthrough.
 
 + You will also need the name of the Azure storage account that you used to create the knowledge store, along with its access key from the Azure portal.
 
@@ -33,7 +33,7 @@ In this article, you'll learn by example how to connect to and explore a knowled
 
 1. Expand the **TABLES** list to show a list of Azure table projections that were created when you ran the **Import Data** wizard on your hotel reviews sample data.
 
-Select any table to view the enriched data, including key phrases sentiment scores, latitude and longitude location data and more.
+Select any table to view the enriched data, including key phrases and sentiment scores.
 
    ![View tables in Storage Explorer](media/knowledge-store-view-storage-explorer/storage-explorer-tables.png "View tables in Storage Explorer")