|
| 1 | +--- |
| 2 | +title: 'Form Recognizer custom skill (C#)' |
| 3 | +titleSuffix: Azure Cognitive Search |
| 4 | +description: Learn how to create a Form Recognizer custom skill using C# and Visual Studio. |
| 5 | + |
| 6 | +manager: nitinme |
| 7 | +author: PatrickFarley |
| 8 | +ms.author: pafarley |
| 9 | +ms.service: cognitive-search |
| 10 | +ms.topic: article |
| 11 | +ms.date: 01/21/2020 |
| 12 | +--- |
| 13 | + |
| 14 | +# Example: Create a Form Recognizer custom skill |
| 15 | + |
| 16 | +In this Azure Cognitive Search skillset example, you'll learn how to create a Form Recognizer custom skill using C# and Visual Studio. Form Recognizer analyzes documents and extracts key/value pairs and table data. By wrapping Form Recognizer into the [custom skill interface](cognitive-search-custom-skill-interface.md), you can add this capability as a step in an end-to-end enrichment pipeline. The pipeline can then load the documents and do other transformations. |
| 17 | + |
| 18 | +## Prerequisites |
| 19 | + |
| 20 | +- [Visual Studio 2019](https://visualstudio.microsoft.com/downloads/) (any edition). |
| 21 | +- At least five forms of the same type. You can use sample data provided with this guide. |
| 22 | + |
| 23 | +## Create a Form Recognizer resource |
| 24 | + |
| 25 | +[!INCLUDE [create resource](../cognitive-services/form-recognizer/includes/create-resource.md)] |
| 26 | + |
| 27 | +## Train your model |
| 28 | + |
| 29 | +You'll need to train a Form Recognizer model with your input forms before you use this skill. Follow the [cURL quickstart](https://docs.microsoft.com/azure/cognitive-services/form-recognizer/quickstarts/curl-train-extract) to learn how to train a model. You can use the sample forms provided in that quickstart, or you can use your own data. Once the model is trained, copy its ID value to a secure location. |
| 30 | + |
| 31 | +## Set up the custom skill |
| 32 | + |
| 33 | +This tutorial uses the [AnalyzeForm](https://github.com/Azure-Samples/azure-search-power-skills/tree/master/Vision/AnalyzeForm) project in the [Azure Search Power Skills](https://github.com/Azure-Samples/azure-search-power-skills) GitHub repository. Clone this repository to your local machine and navigate to **Vision/AnalyzeForm/** to access the project. Then open _AnalyzeForm.csproj_ in Visual Studio. This project creates an Azure Function resource that fulfills the [custom skill interface](cognitive-search-custom-skill-interface.md) and can be used for Azure Cognitive Search enrichment. It takes form documents as inputs, and it outputs (as text) the key/value pairs that you specify. |
| 34 | + |
| 35 | +First, add project-level environment variables. Locate the **AnalyzeForm** project on the left pane, right-click it and select **Properties**. In the **Properties** window, click the **Debug** tab and then find the **Environment variables** field. Click **Add** to add the following variables: |
| 36 | +* `FORMS_RECOGNIZER_ENDPOINT_URL` with the value set to your endpoint URL. |
| 37 | +* `FORMS_RECOGNIZER_API_KEY` with the value set to your subscription key. |
| 38 | +* `FORMS_RECOGNIZER_MODEL_ID` with the value set to the ID of the model you trained. |
| 39 | +* `FORMS_RECOGNIZER_RETRY_DELAY` with the value set to 1000. This value is the time in milliseconds that the program will wait before retrying the query. |
| 40 | +* `FORMS_RECOGNIZER_MAX_ATTEMPTS` with the value set to 100. This value is the number of times the program will query the service while attempting to get a successful response. |
| 41 | + |
| 42 | +Next, open _AnalyzeForm.cs_ and find the `fieldMappings` variable, which references the *field-mappings.json* file. This file (and the variable that references it) defines the list of keys you want to extract from your forms and a custom label for each key. For example, a value of `{ "Address:", "address" }, { "Invoice For:", "recipient" }` means the script will only save the values for the detected `Address:` and `Invoice For:` fields, and it will label those values with `"address"` and `"recipient"`, respectively. |
| 43 | + |
| 44 | +Finally, note the `contentType` variable. This script runs the given Form Recognizer model on remote documents that are referenced by URL, so the content type is `application/json`. If you want to analyze local files by including their byte streams in the HTTP requests, you'll need to change the `contentType` to the appropriate [MIME type](https://developer.mozilla.org/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types) for your file. |
| 45 | + |
| 46 | +## Test the function from Visual Studio |
| 47 | + |
| 48 | +After you've edited your project, save it and set the **AnalyzeForm** project as the startup project in Visual Studio (if it isn't set already). Then press **F5** to run the function in your local environment. Use a REST service like [Postman](https://www.postman.com/) to call the function. |
| 49 | + |
| 50 | +### HTTP request |
| 51 | + |
| 52 | +You'll make the following request to call the function. |
| 53 | + |
| 54 | +```HTTP |
| 55 | +POST https://localhost:7071/api/analyze-form |
| 56 | +``` |
| 57 | + |
| 58 | +### Request body |
| 59 | + |
| 60 | +Start with the request body template below. |
| 61 | + |
| 62 | +```json |
| 63 | +{ |
| 64 | + "values": [ |
| 65 | + { |
| 66 | + "recordId": "record1", |
| 67 | + "data": { |
| 68 | + "formUrl": "<your-form-url>", |
| 69 | + "formSasToken": "<your-sas-token>" |
| 70 | + } |
| 71 | + } |
| 72 | + ] |
| 73 | +} |
| 74 | +``` |
| 75 | + |
| 76 | +Here you'll need to provide the URL of a form that has the same type as the forms you trained with. For testing purposes, you can use one of your training forms. If you followed the cURL quickstart, your forms will be located in an Azure blob storage account. Open Azure Storage Explorer, locate a form file, right-click it, and select **Get Shared Access Signature**. The next dialog window will provide a URL and SAS token. Enter these strings in the `"formUrl"` and `"formSasToken"` fields of your request body, respectively. |
| 77 | + |
| 78 | +> [!div class="mx-imgBorder"] |
| 79 | +>  |
| 80 | +
|
| 81 | +If you want to analyze a remote document that isn't in Azure blob storage, paste its URL in the `"formUrl"` field and leave the `"formSasToken"` field blank. |
| 82 | + |
| 83 | +> [!NOTE] |
| 84 | +> When the skill is integrated in a skillset, the URL and token will be provided by Cognitive Search. |
| 85 | +
|
| 86 | +### Response |
| 87 | + |
| 88 | +You should see a response similar to the following example: |
| 89 | + |
| 90 | +```json |
| 91 | +{ |
| 92 | + "values": [ |
| 93 | + { |
| 94 | + "recordId": "record1", |
| 95 | + "data": { |
| 96 | + "address": "1111 8th st. Bellevue, WA 99501 ", |
| 97 | + "recipient": "Southridge Video 1060 Main St. Atlanta, GA 65024 " |
| 98 | + }, |
| 99 | + "errors": null, |
| 100 | + "warnings": null |
| 101 | + } |
| 102 | + ] |
| 103 | +} |
| 104 | +``` |
| 105 | + |
| 106 | +## Publish the function to Azure |
| 107 | + |
| 108 | +When you're satisfied with the function behavior, you can publish it. |
| 109 | + |
| 110 | +1. In the **Solution Explorer** in Visual Studio, right-click the project and select **Publish**. Choose **Create New** > **Publish**. |
| 111 | +1. If you haven't already connected Visual Studio to your Azure account, select **Add an account....** |
| 112 | +1. Follow the on-screen prompts. Specify a unique name for your app service, the Azure subscription, the resource group, the hosting plan, and the storage account you want to use. You can create a new resource group, a new hosting plan, and a new storage account if you don't already have these. When you're finished, select **Create**. |
| 113 | +1. After the deployment is complete, notice the Site URL. This URL is the address of your function app in Azure. Save it to a temporary location. |
| 114 | +1. In the [Azure portal](https://portal.azure.com), navigate to the Resource Group, and look for the `AnalyzeForm` Function you published. Under the **Manage** section, you should see Host Keys. Copy the *default* host key and save it to a temporary location. |
| 115 | + |
| 116 | +## Connect to your pipeline |
| 117 | + |
| 118 | +To use this skill in a Cognitive Search pipeline, you'll need to add a skill definition to your skillset. The following JSON block is a sample skill definition (you should update the inputs and outputs to reflect your particular scenario and skillset environment). Replace `AzureFunctionEndpointUrl` with your function URL, and replace `AzureFunctionDefaultHostKey` with your host key. |
| 119 | + |
| 120 | +```json |
| 121 | +{ |
| 122 | + "description":"Skillset that invokes the Form Recognizer custom skill", |
| 123 | + "skills":[ |
| 124 | + "[... your existing skills go here]", |
| 125 | + { |
| 126 | + "@odata.type":"#Microsoft.Skills.Custom.WebApiSkill", |
| 127 | + "name":"formrecognizer", |
| 128 | + "description":"Extracts fields from a form using a pre-trained form recognition model", |
| 129 | + "uri":"[AzureFunctionEndpointUrl]/api/analyze-form?code=[AzureFunctionDefaultHostKey]", |
| 130 | + "httpMethod":"POST", |
| 131 | + "timeout":"PT30S", |
| 132 | + "context":"/document", |
| 133 | + "batchSize":1, |
| 134 | + "inputs":[ |
| 135 | + { |
| 136 | + "name":"formUrl", |
| 137 | + "source":"/document/metadata_storage_path" |
| 138 | + }, |
| 139 | + { |
| 140 | + "name":"formSasToken", |
| 141 | + "source":"/document/metadata_storage_sas_token" |
| 142 | + } |
| 143 | + ], |
| 144 | + "outputs":[ |
| 145 | + { |
| 146 | + "name":"address", |
| 147 | + "targetName":"address" |
| 148 | + }, |
| 149 | + { |
| 150 | + "name":"recipient", |
| 151 | + "targetName":"recipient" |
| 152 | + } |
| 153 | + ] |
| 154 | + } |
| 155 | + ] |
| 156 | +} |
| 157 | +``` |
| 158 | + |
| 159 | +## Next steps |
| 160 | + |
| 161 | +In this guide, you created a custom skill from the Azure Form Recognizer service. To learn more about custom skills, see the following resources. |
| 162 | + |
| 163 | +* [Azure Search Power Skills: a repository of custom skills](https://github.com/*zure-Samples/azure-search-power-skills) |
| 164 | +* [Add a custom skill to an AI enrichment pipeline](cognitive-search-custom-skill-interface.md) |
| 165 | +* [Define a skillset](cognitive-search-defining-skillset.md) |
| 166 | +* [Create a skillset (REST)](https://docs.microsoft.com/rest/api/*earchservice/create-skillset) |
| 167 | +* [Map enriched fields](cognitive-search-output-field-mapping.md) |
0 commit comments