Skip to content

Commit 0800d72

Browse files
HeidiSteenHeidiSteen
committed
multimodal RBAC for models
1 parent bbc478b commit 0800d72

4 files changed

+128
-114
lines changed

‎articles/search/tutorial-document-extraction-image-verbalization.md

Lines changed: 28 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ This tutorial demonstrates a lower-cost approach for indexing multimodal content
5151

5252
## Prepare data
5353

54-
The following instructions apply to Azure Storage which provides the sample data and also hosts the knowledge store. A search service identity needs read access to Azure Storage to retrieve the sample data, and it needs write access to create the knowledge store. The search service creates the container for cropped images during skillset processing.
54+
The following instructions apply to Azure Storage which provides the sample data and also hosts the knowledge store. A search service identity needs read access to Azure Storage to retrieve the sample data, and it needs write access to create the knowledge store. The search service creates the container for cropped images during skillset processing, using the name you provide in an environment variable.
5555

5656
1. Download the following sample PDF: [sustainable-ai-pdf](https://cdn-dynmedia-1.microsoft.com/is/content/microsoftcorp/microsoft/msc/documents/presentations/CSR/Accelerating-Sustainability-with-AI-2025.pdf)
5757

@@ -85,7 +85,7 @@ The following instructions apply to Azure Storage which provides the sample data
8585

8686
### Copy a search service URL and API key
8787

88-
For this tutorial, your REST client connection to Azure AI Search requires an endpoint and an API key. You can get these values from the Azure portal. For alternative connection methods, see [Connect to a search service](search-get-started-rbac.md).
88+
For this tutorial, your local REST client connection to Azure AI Search requires an endpoint and an API key. You can get these values from the Azure portal. For alternative connection methods, see [Connect to a search service](search-get-started-rbac.md).
8989

9090
1. Sign in to the [Azure portal](https://portal.azure.com), navigate to the search service **Overview** page, and copy the URL. An example endpoint might look like `https://mydemo.search.windows.net`.
9191

@@ -100,8 +100,8 @@ For this tutorial, your REST client connection to Azure AI Search requires an en
100100
1. Provide values for variables used in the request.
101101

102102
```http
103-
@baseUrl = PUT-YOUR-SEARCH-SERVICE-ENDPOINT-HERE
104-
@apiKey = PUT-YOUR-ADMIN-API-KEY-HERE
103+
@searchUrl = PUT-YOUR-SEARCH-SERVICE-ENDPOINT-HERE
104+
@searchApiKey = PUT-YOUR-ADMIN-API-KEY-HERE
105105
@storageConnection = PUT-YOUR-STORAGE-CONNECTION-STRING-HERE
106106
@openAIResourceUri = PUT-YOUR-OPENAI-URI-HERE
107107
@openAIKey = PUT-YOUR-OPENAI-KEY-HERE
@@ -120,9 +120,9 @@ For help with the REST client, see [Quickstart: Full-text search using REST](sea
120120

121121
```http
122122
### Create a data source
123-
POST {{baseUrl}}/datasources?api-version=2025-05-01-preview HTTP/1.1
123+
POST {{searchUrl}}/datasources?api-version=2025-05-01-preview HTTP/1.1
124124
Content-Type: application/json
125-
api-key: {{apiKey}}
125+
api-key: {{searchApiKey}}
126126
127127
{
128128
"name": "doc-extraction-image-verbalization-ds",
@@ -187,9 +187,9 @@ For nested JSON, the index fields must be identical to the source fields. Curren
187187

188188
```http
189189
### Create an index
190-
POST {{baseUrl}}/indexes?api-version=2025-05-01-preview HTTP/1.1
190+
POST {{searchUrl}}/indexes?api-version=2025-05-01-preview HTTP/1.1
191191
Content-Type: application/json
192-
api-key: {{apiKey}}
192+
api-key: {{searchApiKey}}
193193
194194
{
195195
"name": "doc-extraction-image-verbalization-index",
@@ -296,7 +296,7 @@ POST {{baseUrl}}/indexes?api-version=2025-05-01-preview HTTP/1.1
296296
"azureOpenAIParameters": {
297297
"resourceUri": "{{openAIResourceUri}}",
298298
"deploymentId": "text-embedding-3-large",
299-
"apiKey": "{{openAIKey}}",
299+
"searchApiKey": "{{openAIKey}}",
300300
"modelName": "text-embedding-3-large"
301301
}
302302
}
@@ -339,9 +339,9 @@ The skillset also performs actions specific to images. It uses the GenAI Prompt
339339

340340
```http
341341
### Create a skillset
342-
POST {{baseUrl}}/skillsets?api-version=2025-05-01-preview HTTP/1.1
342+
POST {{searchUrl}}/skillsets?api-version=2025-05-01-preview HTTP/1.1
343343
Content-Type: application/json
344-
api-key: {{apiKey}}
344+
api-key: {{searchApiKey}}
345345
346346
{
347347
"name": "doc-extraction-image-verbalization-skillset",
@@ -419,7 +419,7 @@ POST {{baseUrl}}/skillsets?api-version=2025-05-01-preview HTTP/1.1
419419
],
420420
"resourceUri": "{{openAIResourceUri}}",
421421
"deploymentId": "text-embedding-3-large",
422-
"apiKey": "{{openAIKey}}",
422+
"searchApiKey": "{{openAIKey}}",
423423
"dimensions": 3072,
424424
"modelName": "text-embedding-3-large"
425425
},
@@ -429,7 +429,7 @@ POST {{baseUrl}}/skillsets?api-version=2025-05-01-preview HTTP/1.1
429429
"description": "GenAI Prompt skill for image verbalization",
430430
"uri": "{{chatCompletionResourceUri}}",
431431
"timeout": "PT1M",
432-
"apiKey": "{{chatCompletionKey}}",
432+
"searchApiKey": "{{chatCompletionKey}}",
433433
"context": "/document/normalized_images/*",
434434
"inputs": [
435435
{
@@ -472,7 +472,7 @@ POST {{baseUrl}}/skillsets?api-version=2025-05-01-preview HTTP/1.1
472472
],
473473
"resourceUri": "{{openAIResourceUri}}",
474474
"deploymentId": "text-embedding-3-large",
475-
"apiKey": "{{openAIKey}}",
475+
"searchApiKey": "{{openAIKey}}",
476476
"dimensions": 3072,
477477
"modelName": "text-embedding-3-large"
478478
},
@@ -606,9 +606,9 @@ Key points:
606606

607607
```http
608608
### Create and run an indexer
609-
POST {{baseUrl}}/indexers?api-version=2025-05-01-preview HTTP/1.1
609+
POST {{searchUrl}}/indexers?api-version=2025-05-01-preview HTTP/1.1
610610
Content-Type: application/json
611-
api-key: {{apiKey}}
611+
api-key: {{searchApiKey}}
612612
613613
{
614614
"dataSourceName": "doc-extraction-image-verbalization-ds",
@@ -638,9 +638,9 @@ You can start searching as soon as the first document is loaded.
638638

639639
```http
640640
### Query the index
641-
POST {{baseUrl}}/indexes/doc-extraction-image-verbalization-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
641+
POST {{searchUrl}}/indexes/doc-extraction-image-verbalization-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
642642
Content-Type: application/json
643-
api-key: {{apiKey}}
643+
api-key: {{searchApiKey}}
644644
645645
{
646646
"search": "*",
@@ -689,9 +689,9 @@ Here are some examples of other queries:
689689

690690
```http
691691
### Query for only images
692-
POST {{baseUrl}}/indexes/doc-extraction-image-verbalization-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
692+
POST {{searchUrl}}/indexes/doc-extraction-image-verbalization-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
693693
Content-Type: application/json
694-
api-key: {{apiKey}}
694+
api-key: {{searchApiKey}}
695695
696696
{
697697
"search": "*",
@@ -702,9 +702,9 @@ POST {{baseUrl}}/indexes/doc-extraction-image-verbalization-index/docs/search?ap
702702

703703
```http
704704
### Query for text or images with content related to energy, returning the id, parent document, and text (extracted text for text chunks and verbalized image text for images), and the content path where the image is saved in the knowledge store (only populated for images)
705-
POST {{baseUrl}}/indexes/doc-extraction-image-verbalization-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
705+
POST {{searchUrl}}/indexes/doc-extraction-image-verbalization-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
706706
Content-Type: application/json
707-
api-key: {{apiKey}}
707+
api-key: {{searchApiKey}}
708708
709709
{
710710
"search": "energy",
@@ -719,20 +719,20 @@ Indexers can be reset to clear the high-water mark, which allows a full rerun. T
719719

720720
```http
721721
### Reset the indexer
722-
POST {{baseUrl}}/indexers/doc-extraction-image-verbalization-indexer/reset?api-version=2025-05-01-preview HTTP/1.1
723-
api-key: {{apiKey}}
722+
POST {{searchUrl}}/indexers/doc-extraction-image-verbalization-indexer/reset?api-version=2025-05-01-preview HTTP/1.1
723+
api-key: {{searchApiKey}}
724724
```
725725

726726
```http
727727
### Run the indexer
728-
POST {{baseUrl}}/indexers/doc-extraction-image-verbalization-indexer/run?api-version=2025-05-01-preview HTTP/1.1
729-
api-key: {{apiKey}}
728+
POST {{searchUrl}}/indexers/doc-extraction-image-verbalization-indexer/run?api-version=2025-05-01-preview HTTP/1.1
729+
api-key: {{searchApiKey}}
730730
```
731731

732732
```http
733733
### Check indexer status
734-
GET {{baseUrl}}/indexers/doc-extraction-image-verbalization-indexer/status?api-version=2025-05-01-preview HTTP/1.1
735-
api-key: {{apiKey}}
734+
GET {{searchUrl}}/indexers/doc-extraction-image-verbalization-indexer/status?api-version=2025-05-01-preview HTTP/1.1
735+
api-key: {{searchApiKey}}
736736
```
737737

738738
## Clean up resources

‎articles/search/tutorial-document-extraction-multimodal-embeddings.md

Lines changed: 46 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ This tutorial demonstrates a lower-cost approach for indexing multimodal content
3737
3838
## Prerequisites
3939

40-
+ An [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills). This account provides access to the Azure AI Vision multimodal embedding model used in this tutorial. You must use an Azure AI multi-service account for skillset access to this resource.
40+
+ [Azure AI services multi-service account](/azure/ai-services/multi-service-resource#azure-ai-services-resource-for-azure-ai-search-skills). This account provides access to the Azure AI Vision multimodal embedding model used in this tutorial. You must use an Azure AI multi-service account for skillset access to this resource.
4141

4242
+ [Azure AI Search](search-create-service-portal.md). [Configure your search service](search-manage.md) for role-based access control and a managed identity for connections to Azure Storage and Azure AI Vision. Your service must be on the Basic tier or higher. This tutorial isn't supported on the Free tier. The search service must also be in the same region as your multi-service account.
4343

@@ -51,7 +51,7 @@ This tutorial demonstrates a lower-cost approach for indexing multimodal content
5151

5252
## Prepare data
5353

54-
The following instructions apply to Azure Storage which provides the sample data and also hosts the knowledge store. A search service identity needs read access to Azure Storage to retrieve the sample data, and it needs write access to create the knowledge store. The search service creates the container for cropped images during skillset processing.
54+
The following instructions apply to Azure Storage which provides the sample data and also hosts the knowledge store. A search service identity needs read access to Azure Storage to retrieve the sample data, and it needs write access to create the knowledge store. The search service creates the container for cropped images during skillset processing, using the name you provide in an environment variable.
5555

5656
1. Download the following sample PDF: [sustainable-ai-pdf](https://cdn-dynmedia-1.microsoft.com/is/content/microsoftcorp/microsoft/msc/documents/presentations/CSR/Accelerating-Sustainability-with-AI-2025.pdf)
5757

@@ -83,45 +83,59 @@ The following instructions apply to Azure Storage which provides the sample data
8383
}
8484
```
8585

86-
### Copy a search service URL and API key
86+
## Prepare models
8787

88-
For this tutorial, your REST client connection to Azure AI Search requires an endpoint and an API key. You can get these values from the Azure portal. For alternative connection methods, see [Connect to a search service](search-get-started-rbac.md).
88+
This tutorial assumes you have an existing Azure AI multiservice account through which the skill calls the Azure AI Vision multimodal 4.0 embedding model. The search service connects to the model during skillset processing using its managed identity. This section gives you guidance and links for assigning roles for authorized access.
8989

90-
1. Sign in to the [Azure portal](https://portal.azure.com), navigate to the search service **Overview** page, and copy the URL. An example endpoint might look like `https://mydemo.search.windows.net`.
90+
1. Sign in to the Azure portal (not the Foundry portal) and find the Azure AI multiservice account. Make sure it's in a region that provides the [multimodal 4.0 API](/azure/ai-services/computer-vision/overview-image-analysis#region-availability).
9191

92-
1. Under **Settings** > **Keys**, copy an admin key. Admin keys are used to add, modify, and delete objects. There are two interchangeable admin keys. Copy either one.
92+
1. Select **Access control (IAM)**.
9393

94-
:::image type="content" source="media/search-get-started-rest/get-url-key.png" alt-text="Screenshot of the URL and API keys in the Azure portal.":::
94+
1. Select **Add** and then **Add role assignment**.
95+
96+
1. Search for **Cognitive Services User** and then select it.
97+
98+
1. Choose **Managed identity** and then assign your [search service managed identity](search-howto-managed-identities-data-sources.md).
9599

96100
## Set up your REST file
97101

102+
For this tutorial, your local REST client connection to Azure AI Search requires an endpoint and an API key. You can get these values from the Azure portal. For alternative connection methods, see [Connect to a search service](search-get-started-rbac.md).
103+
104+
For other connections, the search service uses the role assignments you previously defined.
105+
98106
1. Start Visual Studio Code and create a new file.
99107

100108
1. Provide values for variables used in the request.
101109

102110
```http
103-
@baseUrl = PUT-YOUR-SEARCH-SERVICE-ENDPOINT-HERE
104-
@apiKey = PUT-YOUR-ADMIN-API-KEY-HERE
111+
@searchUrl = PUT-YOUR-SEARCH-SERVICE-ENDPOINT-HERE
112+
@searchsearchApiKey = PUT-YOUR-ADMIN-API-KEY-HERE
105113
@storageConnection = PUT-YOUR-STORAGE-CONNECTION-STRING-HERE
106114
@cognitiveServicesUrl = PUT-YOUR-COGNITIVE-SERVICES-URL-HERE
107115
@cognitiveServicesKey= PUT-YOUR-COGNITIVE-SERVICES-URL-KEY-HERE
108116
@modelVersion = PUT-YOUR-VECTORIZE-MODEL-VERSION-HERE
109117
@imageProjectionContainer=PUT-YOUR-IMAGE-PROJECTION-CONTAINER-HERE (Azure AI Search creates this container for you during skills processing)
110118
```
111119

112-
1. Save the file using a `.rest` or `.http` file extension.
120+
1. Save the file using a `.rest` or `.http` file extension. For help with the REST client, see [Quickstart: Full-text search using REST](search-get-started-text.md).
113121

114-
For help with the REST client, see [Quickstart: Full-text search using REST](search-get-started-text.md).
122+
To get the Azure AI Search endpoint and API key:
123+
124+
1. Sign in to the [Azure portal](https://portal.azure.com), navigate to the search service **Overview** page, and copy the URL. An example endpoint might look like `https://mydemo.search.windows.net`.
125+
126+
1. Under **Settings** > **Keys**, copy an admin key. Admin keys are used to add, modify, and delete objects. There are two interchangeable admin keys. Copy either one.
127+
128+
:::image type="content" source="media/search-get-started-rest/get-url-key.png" alt-text="Screenshot of the URL and API keys in the Azure portal.":::
115129

116130
## Create a data source
117131

118132
[Create Data Source (REST)](/rest/api/searchservice/data-sources/create) creates a data source connection that specifies what data to index.
119133

120134
```http
121135
### Create a data source
122-
POST {{baseUrl}}/datasources?api-version=2025-05-01-preview HTTP/1.1
136+
POST {{searchUrl}}/datasources?api-version=2025-05-01-preview HTTP/1.1
123137
Content-Type: application/json
124-
api-key: {{apiKey}}
138+
api-key: {{searchApiKey}}
125139
126140
{
127141
"name": "doc-extraction-multimodal-embedding-ds",
@@ -186,9 +200,9 @@ For nested JSON, the index fields must be identical to the source fields. Curren
186200

187201
```http
188202
### Create an index
189-
POST {{baseUrl}}/indexes?api-version=2025-05-01-preview HTTP/1.1
203+
POST {{searchUrl}}/indexes?api-version=2025-05-01-preview HTTP/1.1
190204
Content-Type: application/json
191-
api-key: {{apiKey}}
205+
api-key: {{searchApiKey}}
192206
193207
{
194208
"name": "doc-extraction-multimodal-embedding-index",
@@ -294,7 +308,7 @@ POST {{baseUrl}}/indexes?api-version=2025-05-01-preview HTTP/1.1
294308
"kind": "aiServicesVision",
295309
"aiServicesVisionParameters": {
296310
"resourceUri": "{{cognitiveServicesUrl}}",
297-
"apiKey": "{{cognitiveServicesKey}}",
311+
"searchApiKey": "{{cognitiveServicesKey}}",
298312
"modelVersion": "{{modelVersion}}"
299313
}
300314
}
@@ -335,9 +349,9 @@ Key points:
335349

336350
```http
337351
### Create a skillset
338-
POST {{baseUrl}}/skillsets?api-version=2025-05-01-preview HTTP/1.1
352+
POST {{searchUrl}}/skillsets?api-version=2025-05-01-preview HTTP/1.1
339353
Content-Type: application/json
340-
api-key: {{apiKey}}
354+
api-key: {{searchApiKey}}
341355
342356
{
343357
"name": "doc-extraction-multimodal-embedding-skillset",
@@ -564,9 +578,9 @@ Key points:
564578

565579
```http
566580
### Create and run an indexer
567-
POST {{baseUrl}}/indexers?api-version=2025-05-01-preview HTTP/1.1
581+
POST {{searchUrl}}/indexers?api-version=2025-05-01-preview HTTP/1.1
568582
Content-Type: application/json
569-
api-key: {{apiKey}}
583+
api-key: {{searchApiKey}}
570584
571585
{
572586
"dataSourceName": "doc-extraction-multimodal-embedding-ds",
@@ -596,9 +610,9 @@ You can start searching as soon as the first document is loaded.
596610

597611
```http
598612
### Query the index
599-
POST {{baseUrl}}/indexes/doc-extraction-multimodal-embedding-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
613+
POST {{searchUrl}}/indexes/doc-extraction-multimodal-embedding-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
600614
Content-Type: application/json
601-
api-key: {{apiKey}}
615+
api-key: {{searchApiKey}}
602616
603617
{
604618
"search": "*",
@@ -644,9 +658,9 @@ For filters, you can also use Logical operators (and, or, not) and comparison op
644658
645659
```http
646660
### Query for only images
647-
POST {{baseUrl}}/indexes/doc-extraction-multimodal-embedding-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
661+
POST {{searchUrl}}/indexes/doc-extraction-multimodal-embedding-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
648662
Content-Type: application/json
649-
api-key: {{apiKey}}
663+
api-key: {{searchApiKey}}
650664
651665
{
652666
"search": "*",
@@ -657,9 +671,9 @@ POST {{baseUrl}}/indexes/doc-extraction-multimodal-embedding-index/docs/search?a
657671

658672
```http
659673
### Query for text or images with content related to energy, returning the id, parent document, and text (only populated for text chunks), and the content path where the image is saved in the knowledge store (only populated for images)
660-
POST {{baseUrl}}/indexes/doc-extraction-multimodal-embedding-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
674+
POST {{searchUrl}}/indexes/doc-extraction-multimodal-embedding-index/docs/search?api-version=2025-05-01-preview HTTP/1.1
661675
Content-Type: application/json
662-
api-key: {{apiKey}}
676+
api-key: {{searchApiKey}}
663677
664678
665679
{
@@ -675,20 +689,20 @@ Indexers can be reset to clear the high-water mark, which allows a full rerun. T
675689

676690
```http
677691
### Reset the indexer
678-
POST {{baseUrl}}/indexers/doc-extraction-multimodal-embedding-indexer/reset?api-version=2025-05-01-preview HTTP/1.1
679-
api-key: {{apiKey}}
692+
POST {{searchUrl}}/indexers/doc-extraction-multimodal-embedding-indexer/reset?api-version=2025-05-01-preview HTTP/1.1
693+
api-key: {{searchApiKey}}
680694
```
681695

682696
```http
683697
### Run the indexer
684-
POST {{baseUrl}}/indexers/doc-extraction-multimodal-embedding-indexer/run?api-version=2025-05-01-preview HTTP/1.1
685-
api-key: {{apiKey}}
698+
POST {{searchUrl}}/indexers/doc-extraction-multimodal-embedding-indexer/run?api-version=2025-05-01-preview HTTP/1.1
699+
api-key: {{searchApiKey}}
686700
```
687701

688702
```http
689703
### Check indexer status
690-
GET {{baseUrl}}/indexers/doc-extraction-multimodal-embedding-indexer/status?api-version=2025-05-01-preview HTTP/1.1
691-
api-key: {{apiKey}}
704+
GET {{searchUrl}}/indexers/doc-extraction-multimodal-embedding-indexer/status?api-version=2025-05-01-preview HTTP/1.1
705+
api-key: {{searchApiKey}}
692706
```
693707

694708
## Clean up resources

0 commit comments

Comments
 (0)