Skip to content

Commit 2f0bd3d

Browse files
aahillaahill
committed
fixes
1 parent 9b76959 commit 2f0bd3d

File tree

5 files changed

+34
-64
lines changed

5 files changed

+34
-64
lines changed

‎articles/ai-services/openai/concepts/use-your-data.md

Lines changed: 0 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -227,24 +227,6 @@ You can modify the following additional settings in the **Data parameters** sect
227227

228228
## Virtual network support & private endpoint support (Azure AI Search only)
229229

230-
See the following table for scenarios supported by virtual networks and private endpoints **when you bring your own Azure AI Search index**.
231-
232-
| Network access to the Azure OpenAI Resource | Network access to the Azure AI Search resource | Is vector search enabled? | Azure OpenAI studio | Chat with the model using the API |
233-
|---------------------------------------------|-------------------------------------------------------|---------------------------|---------------------|-----------------------------------|
234-
| Public | Public | Either | Supported | Supported |
235-
| Private | Public | Yes | Not supported | Supported |
236-
| Private | Public | No | Supported | Supported |
237-
| Regardless of resource access allowances | Private | Either | Not supported | Supported |
238-
239-
Additionally, data ingestion has the following configuration support:
240-
241-
| Network access to the Azure OpenAI Resource | Network access to the Azure AI Search resource | Azure OpenAI studio support | [Ingestion API](../reference.md#start-an-ingestion-job) support |
242-
|---------------------------------------------|-------------------------------------------------------|-----------------------------|-----------------------------------------------------------------|
243-
| Public | Public | Supported | Supported |
244-
| Private | Regardless of resource access allowances. | Not supported | Not supported |
245-
| Public | Private | Not supported | Not supported |
246-
247-
248230

249231
### Azure OpenAI resources
250232

@@ -260,27 +242,6 @@ Learn more about the [manual approval workflow](/azure/private-link/private-endp
260242

261243
After you approve the request in your search service, you can start using the [chat completions extensions API](/azure/ai-services/openai/reference#completions-extensions). Public network access can be disabled for that search service.
262244

263-
### Storage accounts
264-
265-
Storage accounts in virtual networks, firewalls, and private endpoints are supported by Azure OpenAI on your data. To use a storage account in a private network:
266-
267-
1. Ensure you have the system assigned managed identity principal enabled for your Azure OpenAI and Azure AI Search resources.
268-
1. Using the Azure portal, navigate to your resource, and select **Identity** from the navigation menu on the left side of the screen.
269-
1. Set **Status** to **On**.
270-
1. Perform these steps for both of your Azure OpenAI and Azure AI Search resources.
271-
272-
:::image type="content" source="../media/use-your-data/managed-identity.png" alt-text="A screenshot showing managed identity settings in the Azure portal." lightbox="../media/use-your-data/managed-identity.png":::
273-
274-
1. Navigate back to your storage account. Select **Access Control (IAM)** for your resource. Select **Add**, then **Add role assignment**. In the window that appears, add the **Storage Data Contributor** role to the storage resource for your Azure OpenAI and search resource's managed identity.
275-
1. Assign access to **Managed Identity**.
276-
1. If you have multiple search resources, Perform this step for each search resource.
277-
278-
:::image type="content" source="../media/use-your-data/add-role-assignment.png" alt-text="A screenshot showing the role assignment option in the Azure portal." lightbox="../media/use-your-data/add-role-assignment.png":::
279-
280-
1. If your storage account hasn't already been network restricted, go to networking tab and select **Enabled from selected virtual networks and IP addresses**.
281-
282-
:::image type="content" source="../media/use-your-data/enable-virtual-network.png" alt-text="A screenshot showing the option for enabling virtual networks in the Azure portal." lightbox="../media/use-your-data/enable-virtual-network.png":::
283-
284245
## Azure Role-based access controls (Azure RBAC)
285246

286247
To add a new data source to your Azure OpenAI resource, you need the following Azure RBAC roles.
@@ -297,8 +258,6 @@ To add a new data source to your Azure OpenAI resource, you need the following A
297258
| [Cognitive Services Contributor Role](/azure/role-based-access-control/built-in-roles#cognitive-services-contributor) | The Azure AI Search resource, to access Azure OpenAI resource. | You want to deploy a [web app](#using-the-web-app). |
298259

299260

300-
301-
302261
## Document-level access control (Azure AI Search only)
303262

304263
Azure OpenAI on your data lets you restrict the documents that can be used in responses for different users with Azure AI Search [security filters](/azure/search/search-security-trimming-for-azure-search-with-aad). When you enable document level access, the search results returned from Azure AI Search and used to generate a response will be trimmed based on user Microsoft Entra group membership. You can only enable document-level access on existing Azure AI Search indexes. To enable document-level access:

‎articles/ai-services/openai/how-to/use-your-data-securely.md

Lines changed: 34 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -14,23 +14,26 @@ recommendations: false
1414

1515
# Securely use Azure OpenAI on your data
1616

17-
Use this article learn how to use Azure OpenAI on your data securely, for example with virtual networks and private endpoints.
17+
Use this article to learn how to use Azure OpenAI on Your Data securely by protecting data with virtual networks and private endpoints.
1818

1919
## Data ingestion architecture
2020

21-
When you ingest data into Azure OpenAI on your data, the following process is used to process the data and store it in blob storage.
21+
When you ingest data into Azure OpenAI on your data, the following process is used to process the data and store it in blob storage. This applies to the following data sources:
22+
* local files
23+
* Azure blob storage
24+
* URLs
2225

2326
:::image type="content" source="../media/use-your-data/ingestion-architecture.png" alt-text="A diagram showing the process of ingesting data." lightbox="../media/use-your-data/ingestion-architecture.png":::
2427

2528
1. The ingestion process is started when a client sends data to be processed.
26-
1. Ingestion assets (indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) and container in the search resource) are created in Azure AI Search resource and Azure storage account.
27-
1. If the ingestion is triggered by a [scheduled refresh](../concepts/use-your-data.md#schedule-automatic-index-refreshes-azure-ai-search-only), `[3]` is triggered before the above steps.
29+
1. Ingestion assets (indexers, indexes, data sources, a [custom skill](/azure/search/cognitive-search-custom-skill-interface) and container in the search resource) are created in the Azure AI Search resource and Azure blob storage.
30+
1. If the ingestion is triggered by a [scheduled refresh](../concepts/use-your-data.md#schedule-automatic-index-refreshes-azure-ai-search-only), the ingestion process starts at `[3]`.
2831
1. Azure OpenAI's `preprocessing-jobs` API implements the [Azure AI Search customer skill web API protocal](https://learn.microsoft.com/en-us/azure/search/cognitive-search-custom-skill-web-api), and processes the documents in a queue.
2932
1. Azure OpenAI:
30-
1. Cracks the documents.
33+
1. Internally uses the indexer created earlier or Document Intelligence (Private preview only) to crack the documents.
3134
1. Uses a heuristic-based algorithm to perform chunking, honoring table layouts and other formatting elements in the chunk boundary to ensure the best chunking quality.
32-
1. Uses the current embedding model to vectorize the chunks, if `embeddingDeploymentName` is specified in the request header.
33-
1. Azure OpenAI triggers another indexer.
35+
1. If you choose to enable vector search, uses the current embedding model to vectorize the chunks, if `embeddingDeploymentName` is specified in the request header.
36+
1. When all the data that the service is monitoring are processed, Azure OpenAI triggers another indexer.
3437
1. The indexer stores the processed data into an Azure AI Search service.
3538

3639
For the managed identities used in service calls, only system assigned managed identities are supported. User assigned managed identities are not supported.
@@ -39,15 +42,15 @@ For the managed identities used in service calls, only system assigned managed i
3942

4043
:::image type="content" source="../media/use-your-data/inference-architecture.png" alt-text="A diagram showing the process of using the inference API." lightbox="../media/use-your-data/inference-architecture.png":::
4144

42-
When you send API calls to chat with an Azure OpenAI model on your data, the service needs to retrieve the index fields during inference to perform fields mapping automatically if the fields mapping is not explicitly set in the request. That's why the service requires the Azure OpenAI identity to have the `Search Service Contributor` role for the search service even during inference.
45+
When you send API calls to chat with an Azure OpenAI model on your data, the service needs to retrieve the index fields during inference to perform fields mapping automatically if the fields mapping is not explicitly set in the request. Therefore the service requires the Azure OpenAI identity to have the `Search Service Contributor` role for the search service even during inference.
4346

4447

4548

4649
## Resources setup
4750

4851
Use the following sections to set your resources for secure usage.
4952

50-
## Azure OpenAI
53+
## Security support for Azure OpenAI
5154

5255

5356
### Inbound security: networking
@@ -74,18 +77,22 @@ If you use [Azure Management REST API](/rest/api/cognitiveservices/accountmanage
7477
...
7578
```
7679

77-
To use Azure OpenAI Studio, you cannot set `publicNetworkAccess` as `Disabled`, because you need to add your local IP to the IP rules, so Azure OpenAI Studio can call the Azure OpenAI API for both ingestion and inference from your browser.
80+
> [!NOTE]
81+
> To use Azure OpenAI Studio, you cannot set `publicNetworkAccess` as `Disabled`, because you need to add your local IP to the IP rules, so Azure OpenAI Studio can call the Azure OpenAI API for both ingestion and inference from your browser.
7882
7983
### Inbound security: trusted service
8084

81-
To allow Azure AI Search to call Azure OpenAI, `preprocessing-jobs` is used as a custom skill web API (see step 4 in the [data ingestion architecture](#data-ingestion-architecture) section). While Azure OpenAI is network restricted, you will need to setup Azure OpenAI to bypass Azure AI Search as a trusted service. Azure OpenAI will identify the traffic from Azure AI Search by verifying the claims in the JSON Web Token (JWT), so Azure AI Search must use the system assigned managed identity authentication to call the custom skill web API. Set `networkAcls.bypass` as `AzureServices` from the management API. See [Virtual networks article](/azure/ai-services/cognitive-services-virtual-networks?tabs=portal#grant-access-to-trusted-azure-services-for-azure-openai) for more information.
85+
To allow Azure AI Search to call Azure OpenAI `preprocessing-jobs` as custom skill web API, while Azure OpenAI is network restricted, you will need to setup Azure OpenAI to bypass Azure AI Search as a trusted service. Azure OpenAI will identify the traffic from Azure AI Search by verifying the claims in the JSON Web Token (JWT), so Azure AI Search must use the system assigned managed identity authentication to call the custom skill web API. Set `networkAcls.bypass` as `AzureServices` from the management API. See [Virtual networks article](/azure/ai-services/cognitive-services-virtual-networks?tabs=portal#grant-access-to-trusted-azure-services-for-azure-openai) for more information.
8286

8387
### Outbound security: managed identity
8488

8589
To allow other services to recognize Azure OpenAI via Azure Active Directory (Azure AD) authentication, you need to assign a managed identity for your Azure OpenAI service. The easiest way is to toggle on system assigned managed identity on Azure Portal.
8690
:::image type="content" source="../media/use-your-data/openai-managed-identity.png" alt-text="A screenshot showing the system assigned managed identity option in the Azure portal." lightbox="../media/use-your-data/openai-managed-identity.png":::
8791

88-
You can also add a user assigned managed identity, but using user assigned managed identities is only supported by the inference API, not in the ingestion API. Unless you are in an advanced stage of development and ready for production, we recommend using the system assigned managed identity.
92+
You can also add a user assigned managed identity, but using user assigned managed identities is only supported by the inference API, not in the ingestion API.
93+
94+
> [!TIP]
95+
> Unless you are in an advanced stage of development and ready for production, we recommend using the system assigned managed identity.
8996
9097
To set the managed identities via the management API, see [the management API reference documentation](/rest/api/cognitiveservices/accountmanagement/accounts/update#identity).
9198

@@ -101,7 +108,7 @@ To set the managed identities via the management API, see [the management API re
101108
}
102109
```
103110

104-
## Azure AI Search
111+
## Security support for Azure AI Search
105112

106113
### Inbound security: authentication
107114
As Azure OpenAI will use managed identity to access Azure AI Search, you need to enable Azure AD based authentication in your Azure AI Search. To do it on Azure portal, select **Both** in the **Keys** tab in the Azure portal.
@@ -119,13 +126,17 @@ To enable AAD via REST API, set `authOptions` as `aadOrApiKey`. See the [Azure A
119126
}
120127
```
121128

122-
To use Azure OpenAI Studio, you cannot disable the API key based authentication for Azure AI Search, because Azure OpenAI Studio uses the API key to call the Azure AI Search API from your browser. For the best security, when you are ready for production and no longer need to use Azure OpenAI Studio for testing, we recommend that you disable the API key. See the [Azure AI Search RBAC article](/azure/search/search-security-rbac?tabs=config-svc-portal%2Croles-portal%2Ctest-portal%2Ccustom-role-portal%2Cdisable-keys-portal#disable-api-key-authentication) for details.
129+
To use Azure OpenAI Studio, you cannot disable the API key based authentication for Azure AI Search, because Azure OpenAI Studio uses the API key to call the Azure AI Search API from your browser.
130+
131+
> [!TIP]
132+
> For the best security, when you are ready for production and no longer need to use Azure OpenAI Studio for testing, we recommend that you disable the API key. See the [Azure AI Search RBAC article](/azure/search/search-security-rbac?tabs=config-svc-portal%2Croles-portal%2Ctest-portal%2Ccustom-role-portal%2Cdisable-keys-portal#disable-api-key-authentication) for details.
123133
124134
### Inbound security: networking
125135

126-
Use **Selected networks** in the Azure portal. Azure AI Search doesn't support bypassing trusted services, so it is the most complex part in the setup. Create a private endpoint for theAzure OpenAI on your data (as a multi-tenant service managed by Microsoft), and link it to your Azure AI Search resource. This requires you to submit an [application form](/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search#azure-ai-search-resources).
136+
Use **Selected networks** in the Azure portal. Azure AI Search doesn't support bypassing trusted services, so it is the most complex part in the setup. Create a private endpoint for theAzure OpenAI on your data (as a multi-tenant service managed by Microsoft), and link it to your Azure AI Search resource. This requires you to submit an [application form](https://aka.ms/applyacsvpnaoaioyd).
127137

128-
To use Azure OpenAI Studio, you cannot disable public network access, and you need to add your local IP to the IP rules, because Azure AI Studio calls the search API from your browser to list available indexes.
138+
> [!NOTE]
139+
> To use Azure OpenAI Studio, you cannot disable public network access, and you need to add your local IP to the IP rules, because Azure AI Studio calls the search API from your browser to list available indexes.
129140
130141
:::image type="content" source="../media/use-your-data/inbound-networking-security-azure-search.png" alt-text="A screenshot showing the network security for Azure AI search in the Azure portal." lightbox="../media/use-your-data/inbound-networking-security-azure-search.png":::
131142

@@ -137,7 +148,7 @@ To allow other services to recognize the Azure AI Search using Azure AD authenti
137148

138149
User assigned managed identities are not supported.
139150

140-
## Storage account
151+
## Security support for Azure blob storage
141152

142153
### Inbound security: networking
143154
In the Azure portal, navigate to your storage account networking tab and select **Enabled from selected virtual networks and IP addresses**.
@@ -176,7 +187,7 @@ Also, make sure that the IP your development machine is whitelisted in the IP ru
176187

177188
:::image type="content" source="../media/use-your-data/ip-rules-azure-portal.png" alt-text="A screenshot showing roles for accounts in the Azure portal." lightbox="../media/use-your-data/ip-rules-azure-portal.png":::
178189

179-
## Ingestion API
190+
### Ingestion API
180191

181192

182193
See the [ingestion API reference article](/azure/ai-services/openai/reference#start-an-ingestion-job) for details on the request and response objects used by the ingestion API.
@@ -219,7 +230,7 @@ curl -i -X GET https://wednesday-tip-vnet.openai.azure.com/openai/extensions/on-
219230
-H "Authorization: Bearer $accessToken"
220231
```
221232

222-
## Inference API
233+
### Inference API
223234

224235
See the [inference API reference article](/azure/ai-services/openai/reference#completions-extensions) for details on the request and response objects used by the inference API.
225236

@@ -232,7 +243,7 @@ Example:
232243

233244
```bash
234245
accessToken=$(az account get-access-token --resource https://cognitiveservices.azure.com/ --query "accessToken" --output tsv)
235-
curl -i -X POST https://wednesday-test-usnc.openai.azure.com/openai/deployments/turbo/extensions/chat/completions?api-version=2023-10-01-preview \
246+
curl -i -X POST https://my-resource.openai.azure.com/openai/deployments/turbo/extensions/chat/completions?api-version=2023-10-01-preview \
236247
-H "Content-Type: application/json" \
237248
-H "Authorization: Bearer $accessToken" \
238249
-d \
@@ -242,7 +253,7 @@ curl -i -X POST https://wednesday-test-usnc.openai.azure.com/openai/deployments/
242253
{
243254
"type": "AzureCognitiveSearch",
244255
"parameters": {
245-
"endpoint": "https://wednesday-test.search.windows.net",
256+
"endpoint": "https://my-search-service.search.windows.net",
246257
"indexName": "unit-test-vector-index",
247258
"queryType": "vector",
248259
"embeddingDeploymentName": "ada"
@@ -259,11 +270,11 @@ curl -i -X POST https://wednesday-test-usnc.openai.azure.com/openai/deployments/
259270
'
260271
```
261272

262-
# Azure OpenAI Studio
273+
## Azure OpenAI Studio
263274

264275
You should be able to use all Azure OpenAI Studio features, including both ingestion and inference.
265276

266-
# Web app
277+
## Web app
267278
The web app published from the Studio will communicate with Azure OpenAI. If Azure OpenAI is network restricted, the web app need to be setup correctly for outbound networking.
268279

269280
1. Set Azure OpenAI allow inbound traffic from your virtual network.

‎articles/ai-services/openai/media/use-your-data/add-role-assignment.png

-28.6 KB
Binary file not shown.

‎articles/ai-services/openai/media/use-your-data/enable-virtual-network.png

-90.4 KB
Binary file not shown.

‎articles/ai-services/openai/media/use-your-data/managed-identity.png

-112 KB
Binary file not shown.

0 commit comments

Comments
 (0)