You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/cognitive-search-custom-skill-web-api.md
+22-22Lines changed: 22 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,17 +8,17 @@ ms.service: cognitive-search
8
8
ms.custom:
9
9
- ignite-2023
10
10
ms.topic: conceptual
11
-
ms.date: 12/18/2023
11
+
ms.date: 03/05/2024
12
12
---
13
13
14
14
# Custom Web API skill in an Azure AI Search enrichment pipeline
15
15
16
-
The **Custom Web API** skill allows you to extend AI enrichment by calling out to a Web API endpoint providing custom operations. Similar to built-in skills, a **Custom Web API** skill has inputs and outputs. Depending on the inputs, your Web API receives a JSON payload when the indexer runs, and outputs a JSON payload as a response, along with a success status code. The response is expected to have the outputs specified by your custom skill. Any other response is considered an error and no enrichments are performed.
16
+
The **Custom Web API** skill allows you to extend AI enrichment by calling out to a Web API endpoint providing custom operations. Similar to built-in skills, a **Custom Web API** skill has inputs and outputs. Depending on the inputs, your Web API receives a JSON payload when the indexer runs, and outputs a JSON payload as a response, along with a success status code. The response is expected to have the outputs specified by your custom skill. Any other response is considered an error and no enrichments are performed. The structure of the JSON payload is described further down in this document.
17
17
18
-
The structure of the JSON payload is described further down in this document.
18
+
The **Custom Web API** skill is also used in the implementation of [Azure OpenAI On Your Data](/azure/ai-services/openai/concepts/use-your-data) feature. If Azure OpenAI is [configured for role-based access](/azure/ai-services/openai/how-to/use-your-data-securely#configure-azure-openai) and you get `403 Forbidden` calls when creating the vector store, verify that Azure AI Search has a [system assigned identity](search-howto-managed-identities-data-sources.md#create-a-system-managed-identity) and runs as a [trusted service](/azure/ai-services/openai/how-to/use-your-data-securely#enable-trusted-service) on Azure OpenAI.
19
19
20
20
> [!NOTE]
21
-
> The indexer will retry twice for certain standard HTTP status codes returned from the Web API. These HTTP status codes are:
21
+
> The indexer retries twice for certain standard HTTP status codes returned from the Web API. These HTTP status codes are:
22
22
> *`502 Bad Gateway`
23
23
> *`503 Service Unavailable`
24
24
> *`429 Too Many Requests`
@@ -33,13 +33,13 @@ Parameters are case-sensitive.
33
33
34
34
| Parameter name | Description |
35
35
|--------------------|-------------|
36
-
|`uri`| The URI of the Web API to which the JSON payload will be sent. Only the **https** URI scheme is allowed. |
37
-
|`authResourceId`| (Optional) A string that if set, indicates that this skill should use a managed identity on the connection to the function or app hosting the code. This property takes an application (client) ID or app's registration in Microsoft Entra ID, in a [supported format](../active-directory/develop/security-best-practices-for-app-registration.md#application-id-uri): `api://<appId>`. This value will be used to scope the authentication token retrieved by the indexer, and will be sent along with the custom Web skill API request to the function or app. Setting this property requires that your search service is [configured for managed identity](search-howto-managed-identities-data-sources.md) and your Azure function app is [configured for a Microsoft Entra login](../app-service/configure-authentication-provider-aad.md). To use this parameter, call the API with `api-version=2023-10-01-Preview`. |
36
+
|`uri`| The URI of the Web API to which the JSON payload is sent. Only the **https** URI scheme is allowed. |
37
+
|`authResourceId`| (Optional) A string that if set, indicates that this skill should use a managed identity on the connection to the function or app hosting the code. This property takes an application (client) ID or app's registration in Microsoft Entra ID, in a [supported format](../active-directory/develop/security-best-practices-for-app-registration.md#application-id-uri): `api://<appId>`. This value is used to scope the authentication token retrieved by the indexer, and is sent along with the custom Web skill API request to the function or app. Setting this property requires that your search service is [configured for managed identity](search-howto-managed-identities-data-sources.md) and your Azure function app is [configured for a Microsoft Entra sign in](../app-service/configure-authentication-provider-aad.md). To use this parameter, call the API with `api-version=2023-10-01-Preview`. |
38
38
|`httpMethod`| The method to use while sending the payload. Allowed methods are `PUT` or `POST`|
39
-
|`httpHeaders`| A collection of key-value pairs where the keys represent header names and values represent header values that will be sent to your Web API along with the payload. The following headers are prohibited from being in this collection: `Accept`, `Accept-Charset`, `Accept-Encoding`, `Content-Length`, `Content-Type`, `Cookie`, `Host`, `TE`, `Upgrade`, `Via`. |
39
+
|`httpHeaders`| A collection of key-value pairs where the keys represent header names and values represent header values that are sent to your Web API along with the payload. The following headers are prohibited from being in this collection: `Accept`, `Accept-Charset`, `Accept-Encoding`, `Content-Length`, `Content-Type`, `Cookie`, `Host`, `TE`, `Upgrade`, `Via`. |
40
40
|`timeout`| (Optional) When specified, indicates the timeout for the http client making the API call. It must be formatted as an XSD "dayTimeDuration" value (a restricted subset of an [ISO 8601 duration](https://www.w3.org/TR/xmlschema11-2/#dayTimeDuration) value). For example, `PT60S` for 60 seconds. If not set, a default value of 30 seconds is chosen. The timeout can be set to a maximum of 230 seconds and a minimum of 1 second. |
41
-
|`batchSize`| (Optional) Indicates how many "data records" (see JSON payload structure below) will be sent per API call. If not set, a default of 1000 is chosen. We recommend that you make use of this parameter to achieve a suitable tradeoff between indexing throughput and load on your API. |
42
-
|`degreeOfParallelism`| (Optional) When specified, indicates the number of calls the indexer will make in parallel to the endpoint you have provided. You can decrease this value if your endpoint is failing under pressure, or raise it if your endpoint can handle the load. If not set, a default value of 5 is used. The `degreeOfParallelism` can be set to a maximum of 10 and a minimum of 1. |
41
+
|`batchSize`| (Optional) Indicates how many "data records" (see JSON payload structure below) is sent per API call. If not set, a default of 1000 is chosen. We recommend that you make use of this parameter to achieve a suitable tradeoff between indexing throughput and load on your API. |
42
+
|`degreeOfParallelism`| (Optional) When specified, indicates the number of calls the indexer makes in parallel to the endpoint you provide. You can decrease this value if your endpoint is failing under pressure, or raise it if your endpoint can handle the load. If not set, a default value of 5 is used. The `degreeOfParallelism` can be set to a maximum of 10 and a minimum of 1. |
43
43
44
44
## Skill inputs
45
45
@@ -82,16 +82,16 @@ There are no predefined outputs for this skill. Be sure to [define an output fie
82
82
83
83
## Sample input JSON structure
84
84
85
-
This JSON structure represents the payload that will be sent to your Web API.
86
-
It will always follow these constraints:
85
+
This JSON structure represents the payload that is sent to your Web API.
86
+
It always follows these constraints:
87
87
88
-
* The top-level entity is called `values` and will be an array of objects. The number of such objects will be at most the `batchSize`.
88
+
* The top-level entity is called `values` and is an array of objects. The number of such objects are at most the `batchSize`.
89
89
90
-
* Each object in the `values` array will have:
90
+
* Each object in the `values` array has:
91
91
92
92
* A `recordId` property that is a **unique** string, used to identify that record.
93
93
94
-
* A `data` property that is a JSON object. The fields of the `data` property will correspond to the "names" specified in the `inputs` section of the skill definition. The value of those fields will be from the `source` of those fields (which could be from a field in the document, or potentially from another skill).
94
+
* A `data` property that is a JSON object. The fields of the `data` property corresponds to the "names" specified in the `inputs` section of the skill definition. The values of those fields are from the `source` of those fields (which could be from a field in the document, or potentially from another skill).
95
95
96
96
```json
97
97
{
@@ -140,7 +140,7 @@ It will always follow these constraints:
140
140
141
141
The "output" corresponds to the response returned from your Web API. The Web API should only return a JSON payload (verified by looking at the `Content-Type` response header) and should satisfy the following constraints:
142
142
143
-
* There should be a top-level entity called `values` which should be an array of objects.
143
+
* There should be a top-level entity called `values`, which should be an array of objects.
144
144
145
145
* The number of objects in the array should be the same as the number of objects sent to the Web API.
146
146
@@ -150,11 +150,11 @@ The "output" corresponds to the response returned from your Web API. The Web API
150
150
151
151
* A `data` property, which is an object where the fields are enrichments matching the "names" in the `output` and whose value is considered the enrichment.
152
152
153
-
* An `errors` property, an array listing any errors encountered that will be added to the indexer execution history. This property is required, but can have a `null` value.
153
+
* An `errors` property, an array listing any errors encountered that is added to the indexer execution history. This property is required, but can have a `null` value.
154
154
155
-
* A `warnings` property, an array listing any warnings encountered that will be added to the indexer execution history. This property is required, but can have a `null` value.
155
+
* A `warnings` property, an array listing any warnings encountered that is added to the indexer execution history. This property is required, but can have a `null` value.
156
156
157
-
* The ordering of objects in the `values` in either the request or response isn't important. However, the `recordId` is used for correlation so any record in the response containing a `recordId` which was not part of the original request to the Web API will be discarded.
157
+
* The ordering of objects in the `values` in either the request or response isn't important. However, the `recordId` is used for correlation so any record in the response containing a `recordId`, which wasn't part of the original request to the Web API is discarded.
158
158
159
159
```json
160
160
{
@@ -207,15 +207,15 @@ The "output" corresponds to the response returned from your Web API. The Web API
207
207
208
208
In addition to your Web API being unavailable, or sending out non-successful status codes the following are considered erroneous cases:
209
209
210
-
* If the Web API returns a success status code but the response indicates that it is not `application/json` then the response is considered invalid and no enrichments will be performed.
210
+
* If the Web API returns a success status code but the response indicates that it isn't `application/json` then the response is considered invalid and no enrichments are performed.
211
211
212
-
* If there are invalid records (for example, `recordId` is missing or duplicated) in the response `values` array, no enrichment will be performed for the invalid records. It's important to adhere to the Web API skill contract when developing custom skills. You can refer to [this example](https://github.com/Azure-Samples/azure-search-power-skills/blob/main/Common/WebAPISkillContract.cs) provided in the [Power Skill repository](https://github.com/Azure-Samples/azure-search-power-skills/tree/main) that follows the expected contract.
212
+
* If there are invalid records (for example, `recordId` is missing or duplicated) in the response `values` array, no enrichment is performed for the invalid records. It's important to adhere to the Web API skill contract when developing custom skills. You can refer to [this example](https://github.com/Azure-Samples/azure-search-power-skills/blob/main/Common/WebAPISkillContract.cs) provided in the [Power Skill repository](https://github.com/Azure-Samples/azure-search-power-skills/tree/main) that follows the expected contract.
213
213
214
-
For cases when the Web API is unavailable or returns an HTTP error, a friendly error with any available details about the HTTP error will be added to the indexer execution history.
214
+
For cases when the Web API is unavailable or returns an HTTP error, a friendly error with any available details about the HTTP error is added to the indexer execution history.
215
215
216
216
## See also
217
217
218
-
+[How to define a skillset](cognitive-search-defining-skillset.md)
218
+
+[Define a skillset](cognitive-search-defining-skillset.md)
219
219
+[Add custom skill to an AI enrichment pipeline](cognitive-search-custom-skill-interface.md)
220
220
+[Example: Creating a custom skill for AI enrichment](cognitive-search-create-custom-skill-example.md)
Copy file name to clipboardExpand all lines: articles/search/search-manage.md
+11-5Lines changed: 11 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,15 +25,21 @@ In Azure AI Search, the [Azure portal](https://portal.azure.com) supports a broa
25
25
26
26
Each search service is managed as a standalone resource. Your role assignment determines what operations are exposed in the portal.
27
27
28
-
## Administrator permissions
28
+
## Portal and administrator permissions
29
29
30
30
Portal access is through [role assignments](search-security-rbac.md). By default, all search services start with at least one Owner. Owners, service administrators, and co-administrators have permission to create other administrators and other role assignments. They have full access to all portal pages and operations.
31
31
32
32
Contributors and Search Service Contributors have the same access as Owner, minus the ability to assign roles.
33
33
34
34
Readers have access to service information in the Essentials section and in the Monitoring tab. Access is limited. A reader can get basic information about a search service, but not enough to set up a connection or confirm the existence of objects on the service.
35
35
36
-
For data plane tasks, the portal attempts admin API keys first. If keys are disabled, the operation succeeds if Search Index Data Contributor and Search Index Data Reader roles provide permission.
36
+
For data plane tasks, such as creating and configuring indexes and indexers: on a default system, the portal attempts admin API keys first, even if there are role assignments. If [keys are disabled](search-security-rbac.md#disable-api-key-authentication), here's the portal experience for the following roles:
37
+
38
+
* Search Index Data Contributor can see the list of indexers, and access an individual one to see its historical runs and status, but cannot run, reset, create, update, or delete it.
39
+
40
+
* A Search Index Data Reader can query the indexes.
41
+
42
+
In short, if you want unrestricted access to portal features, including the ability to run the Import data wizards, you should have Contributor or Search Servicer Contributor permissions.
37
43
38
44
> [!TIP]
39
45
> By default, any owner or administrator can create or delete services. To prevent accidental deletions, you can [lock resources](../azure-resource-manager/management/lock-resources.md).
@@ -48,7 +54,7 @@ The overview page is the "home" page of each service. In the following screensho
48
54
|------|-------------|
49
55
| 1 | A command bar at the top of the page includes [Import data wizard](search-get-started-portal.md) and [Search explorer](search-explorer.md), used for prototyping and exploration. |
50
56
| 2 | The **Essentials** section lists service properties, such as the service endpoint, service tier, and replica and partition counts. |
51
-
| 3 | Tabbed pages in the center provide quick access to usage statistics and service health metrics.|
57
+
| 3 | Tabbed pages in the center provide quick access to usage statistics and service health metrics.|
52
58
| 4 | Navigation links to existing indexes, indexers, data sources, and skillsets. |
53
59
54
60
You can't change the search service name, subscription, resource group, region (location), or tier. Switching tiers requires creating a new service or filing a support ticket to request a tier upgrade, which is only supported for Basic and higher.
@@ -104,9 +110,9 @@ A search service is always created with [API keys](search-security-api-keys.md).
104
110
Developers need the following information to connect to Azure AI Search:
105
111
106
112
+ An endpoint or URL, provided on the **Overview** page.
107
-
+ An API key from the **Keys** page, or a role assignment (Search Index Data Contributor is recommended).
113
+
+ An API key from the **Keys** page, or a role assignment (contributor is recommended).
108
114
109
-
We recommend portal access for the following wizards and tools: [Import data wizard](search-get-started-portal.md), [Import and vectorize data](search-get-started-portal-import-vectors.md), [Search explorer](search-explorer.md).
115
+
We recommend portal access for the following wizards and tools: [Import data wizard](search-get-started-portal.md), [Import and vectorize data](search-get-started-portal-import-vectors.md), [Search explorer](search-explorer.md). Recall that a user must be a contributor or above to run the import wizards.
0 commit comments