checkpoint

Author: HeidiSteen

articles/search/cognitive-search-tutorial-blob-dotnet.md

@@ -1,138 +1,101 @@
 ---
-title: 'C# tutorial: AI on Azure blobs'
+title: 'Tutorial: Skillsets using C#'
 titleSuffix: Azure AI Search
-description: Step through an example of text extraction and natural language processing over content in Blob storage using C# and the Azure AI Search .NET SDK.
-author: gmndrg
-ms.author: gimondra
+description: Use C# and the Azure SDK for .NET to create skillsets. This skillset applies AI transformations and analyses to create searchable content from images and unstructured text.
+
+author: HeidiSteen
+ms.author: heidist
 manager: nitinme
 
 ms.service: cognitive-search
 ms.topic: tutorial
-ms.date: 09/13/2023
+ms.date: 03/06/2024
 ms.custom:
   - devx-track-csharp
   - devx-track-dotnet
   - ignite-2023
 ---
 
-# Tutorial: Use .NET and AI to generate searchable content from Azure blobs
+# C# Tutorial: Use skillsets to generate searchable content in Azure AI Search
+
+In this tutorial, learn how to use the [Azure SDK for .NET](https://www.nuget.org/packages/Azure.Search.Documents/) to create an [AI enrichment pipeline](cognitive-search-concept-intro.md) for content extraction and transformations during indexing.
 
-If you have unstructured text or images in Azure Blob Storage, an [AI enrichment pipeline](cognitive-search-concept-intro.md) in Azure AI Search can extract information and create new content for full-text search or knowledge mining scenarios. 
+Skillsets add AI processing to raw content, making that content more uniform and searchable. Once you know how skillsets work, you can support a broad range of transformations: from image analysis, to natural language processing, to customized processing that you provide externally.
 
-In this C# tutorial, you learn how to:
+This tutorial helps you learn how to:
 
 > [!div class="checklist"]
-> * Set up a development environment.
-> * Define a pipeline that uses OCR, language detection, entity recognition, and key phrase extraction.
-> * Execute the pipeline to invoke transformations, and to create and load a search index.
-> * Explore results using full text search and a rich query syntax.
+> + Define objects in an enrichment pipeline
+> + Build a skillset that invokes OCR, language detection, entity recognition, and key phrase extraction
+> + Execute the pipeline to invoke transformations, and to create and load a search index
+> + Check the results using full text search
 
 If you don't have an Azure subscription, open a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
 
 ## Overview
 
-This tutorial uses C# and the [**Azure.Search.Documents** client library](/dotnet/api/overview/azure/search.documents-readme) to create a data source, index, indexer, and skillset.
+This tutorial uses C# and the [**Azure.Search.Documents**](/dotnet/api/overview/azure/search.documents-readme) client library to create a data source, index, indexer, and skillset.
 
-The indexer connects to sample data in a blob container that's specified in the data source object, and sends all enriched content to a search index.
+The [indexer](search-indexer-overview.md) drives each step in the pipeline, starting with content extraction of sample data (unstructured text and images) in a blob container on Azure Storage.
 
-The skillset is attached to the indexer. It uses built-in skills from Microsoft to find and extract information. Steps in the pipeline include Optical Character Recognition (OCR) on images, language detection on text, key phrase extraction, and entity recognition (organizations). New information created by the pipeline is stored in new fields in an index. Once the index is populated, you can use the fields in queries, facets, and filters.
+Once content is extracted, the [skillset](cognitive-search-working-with-skillsets.md) executes built-in skills from Microsoft to find and extract information. These skills include Optical Character Recognition (OCR) on images, language detection on text, key phrase extraction, and entity recognition (organizations). New information created by the skillset is sent to fields in an [index](search-what-is-an-index.md). Once the index is populated, you can use the fields in queries, facets, and filters.
 
 ## Prerequisites
 
-* [Visual Studio](https://visualstudio.microsoft.com/downloads/)
-* [Azure.Search.Documents 11.x NuGet package](https://www.nuget.org/packages/Azure.Search.Documents) 
-* [Azure Storage](https://azure.microsoft.com/services/storage/)
-* [Azure AI Search](https://azure.microsoft.com/services/search/)
-* [Sample data](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/ai-enrichment-mixed-media)
-
-> [!Note]
-> You can use the free search service for this tutorial. A free search service limits you to three indexes, three indexers, and three data sources. This tutorial creates one of each. Before starting, make sure you have room on your service to accept the new resources.
-
-## Download files
-
-The sample data consists of 14 files of mixed content type that you will upload to Azure Blob Storage in a later step.
-
-1. Get the files from [azure-search-sample-data/ai-enrichment-mixed-media/](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/ai-enrichment-mixed-media) and copy them to your local computer. 
-
-1. Next, get the source code for this tutorial. Source code is in the **tutorial-ai-enrichment/v11** folder in the [azure-search-dotnet-samples](https://github.com/Azure-Samples/azure-search-dotnet-samples) repository.
-
-## 1 - Create services
-
-This tutorial uses Azure AI Search for indexing and queries, Azure AI services on the backend for AI enrichment, and Azure Blob Storage to provide the data. This tutorial stays under the free allocation of 20 transactions per indexer per day on Azure AI services, so the only services you need to create are search and storage.
-
-If possible, create both in the same region and resource group for proximity and manageability. In practice, your Azure Storage account can be in any region.
-
-### Start with Azure Storage
++ [Visual Studio](https://visualstudio.microsoft.com/downloads/)
 
-1. Sign in to the [Azure portal](https://portal.azure.com) and click **+ Create Resource**.
++ [Azure.Search.Documents 11.x NuGet package](https://www.nuget.org/packages/Azure.Search.Documents) 
 
-1. Search for *storage account* and select Microsoft's Storage Account offering.
++ [Azure Storage](/azure/storage/common/storage-account-create)
 
-   ![Create Storage account](media/cognitive-search-tutorial-blob/storage-account.png "Create Storage account")
++ [Azure AI Search](search-create-app-portal.md)
 
-1. In the Basics tab, the following items are required. Accept the defaults for everything else.
++ [Sample data files (mixed media)](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/ai-enrichment-mixed-media)
 
-   + **Resource group**. Select an existing one or create a new one, but use the same group for all services so that you can manage them collectively.
-
-   + **Storage account name**. If you think you might have multiple resources of the same type, use the name to disambiguate by type and region, for example *blobstoragewestus*. 
-
-   + **Location**. If possible, choose the same location used for Azure AI Search and Azure AI services. A single location voids bandwidth charges.
-
-   + **Account Kind**. Choose the default, *StorageV2 (general purpose v2)*.
-
-1. Select **Review + Create** to create the service.
-
-1. Once it's created, select **Go to the resource** to open the Overview page.
+> [!NOTE]
+> You can use the free search service for this tutorial. A free search service limits you to three indexes, three indexers, and three data sources. This tutorial creates one of each. Before starting, make sure you have room on your service to accept the new resources.
 
-1. Select **Blobs** service.
+### Upload sample data to Azure Storage
 
-1. Select **+ Container** to create a container and name it *cog-search-demo*.
+1. In Azure Storage, create a new container and name it *cog-search-demo*.
 
-1. Select *cog-search-demo* and then select **Upload** to open the folder where you saved the download files. Select all of the files. Select **Upload**.
+1. [Upload the sample data files](/azure/storage/blobs/storage-quickstart-blobs-portal).
 
    :::image type="content" source="media/cognitive-search-tutorial-blob/sample-files.png" alt-text="Screenshot of the files in File Explorer." border="true":::
 
-1. Before you leave Azure Storage, get a connection string so that you can formulate a connection in Azure AI Search. 
-
-   1. Browse back to the Overview page of your storage account (we used *blobstragewestus* as an example). 
+1. Get a storage connection string so that you can formulate a connection in Azure AI Search.
 
-   1. In the left navigation pane, select **Access keys** and copy one of the connection strings. 
+   1. On the left, select **Access keys**.
 
-   The connection string is a URL similar to the following example:
+   1. Copy the connection string for either key one or key two. The connection string is similar to the following example:
 
       ```http
       DefaultEndpointsProtocol=https;AccountName=cogsrchdemostorage;AccountKey=<your account key>;EndpointSuffix=core.windows.net
       ```
 
-1. Save the connection string to Notepad. You'll need it later when setting up the data source connection.
-
-<!-- The next section says that a key isn't required, but the code won't run without it. Is there a way to make the key declaration work as null? It would be nice to keep the appsetting so that people know how to set it up, but at the same time, the other versions of this sample don't require a key, so this one shouldn't either. -->
-
 ### Azure AI services
 
-AI enrichment is backed by Azure AI services, including Language service and Azure AI Vision for natural language and image processing. If your objective was to complete an actual prototype or project, you would at this point provision Azure AI services (in the same region as Azure AI Search) so that you can attach it to indexing operations.
-
-For this exercise, however, you can skip resource provisioning because Azure AI Search can connect to Azure AI services behind the scenes and give you 20 free transactions per indexer run. Since this tutorial uses 14 transactions, the free allocation is sufficient. For larger projects, plan on provisioning Azure AI services at the pay-as-you-go S0 tier. For more information, see [Attach Azure AI services](cognitive-search-attach-cognitive-services.md).
+AI enrichment is backed by Azure AI services, including Language service and Azure AI Vision for natural language and image processing. For small workloads like this tutorial, you can use the free allocation of twenty transactions per indexer. For larger workloads, [attach an Azure AI Services multi-region resource to a skillset](cognitive-search-attach-cognitive-services.md) for pay-as-you-go pricing.
 
 ### Azure AI Search
 
 The third component is Azure AI Search, which you can [create in the portal](search-create-service-portal.md) or [find an existing search service](https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.Search%2FsearchServices) in your subscription.
 
-You can use the Free tier to complete this walkthrough. 
+You can use the Free tier to complete this walkthrough.
 
-### Copy an admin api-key and URL for Azure AI Search
+### Copy a key and URL
 
-To interact with your Azure AI Search service you will need the service URL and an access key.
+For this tutorial, connections to Azure AI Search require an endpoint and an API key. You can get these values from the Azure portal.
 
-1. Sign in to the [Azure portal](https://portal.azure.com), and in your search service **Overview** page, get the name of your search service. You can confirm your service name by reviewing the endpoint URL. If your endpoint URL were `https://mydemo.search.windows.net`, your service name would be `mydemo`.
+1. Sign in to the [Azure portal](https://portal.azure.com), navigate to the search service **Overview** page, and copy the URL. An example endpoint might look like `https://mydemo.search.windows.net`.
 
-1. In **Settings** > **Keys**, get an admin key for full rights on the service. You can copy either the primary or secondary key.
+1. Under **Settings** > **Keys**, copy an admin key. Admin keys are used to add, modify, and delete objects. There are two interchangeable admin keys. Copy either one.
 
-   ![Get the service name and admin key](media/search-get-started-javascript/service-name-and-keys.png)
+   :::image type="content" source="media/search-get-started-rest/get-url-key.png" alt-text="Screenshot of the URL and API keys in the Azure portal.":::
 
-Having a valid key establishes trust, on a per request basis, between the application sending the request and the service that handles it.
+A valid API key establishes trust, on a per request basis, between the application sending the request and the search service handling it.
 
-## 2 - Set up your environment
+## Set up your environment
 
 Begin by opening Visual Studio and creating a new Console App project that can run on .NET Core.
 
@@ -232,7 +195,7 @@ private static void ExitProgram(string message)
 }
 ```
 
-## 3 - Create the pipeline
+## Create the pipeline
 
 In Azure AI Search, AI processing occurs during indexing (or data ingestion). This part of the walkthrough creates four objects: data source, index definition, skillset, indexer. 
 
@@ -777,7 +740,7 @@ When content is extracted, you can set `imageAction` to extract text from images
 
 <a name="check-indexer-status"></a>
 
-## 4 - Monitor indexing
+## Monitor indexing
 
 Once the indexer is defined, it runs automatically when you submit the request. Depending on which skills you defined, indexing can take longer than you expect. To find out whether the indexer is still running, use the `GetStatus` method.
 
@@ -823,7 +786,7 @@ Console.WriteLine("Check the indexer overall status...");
 CheckIndexerOverallStatus(indexerClient, demoIndexer);
 ```
 
-## 5 - Search
+## Search
 
 In Azure AI Search tutorial console apps, we typically add a 2-second delay before running queries that return results, but because enrichment takes several minutes to complete, we'll close the console app and use another approach instead.