You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/chat-completion-skill-example-usage.md
+10-10Lines changed: 10 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,11 +2,11 @@
2
2
title: Utilize the content generation capabilities of language models as part of content ingestion pipeline
3
3
titleSuffix: Azure AI Search
4
4
description: Use language models to caption your images and facilitate an image search through your data.
5
-
author: amitkalay
6
-
ms.author: amitkalay
5
+
author: gmndrg
6
+
ms.author: gimondra
7
7
ms.service: azure-ai-search
8
8
ms.topic: how-to
9
-
ms.date: 05/05/2025
9
+
ms.date: 07/28/2025
10
10
ms.custom:
11
11
- devx-track-csharp
12
12
- build-2025
@@ -22,20 +22,20 @@ The GenAI Prompt skill (preview) generates a description of each image in your d
22
22
23
23
To work with image content in a skillset, you need:
24
24
25
-
+ A supported data source
26
-
+ Files or blobs containing images
27
-
+ Read access on the supported data source. This article uses key-based authentication, but indexers can also connect using the search service identity and Microsoft Entra ID authentication. For role-based access control, assign roles on the data source to allow read access by the service identity. If you're testing on a local development machine, make sure you also have read access on the supported data source.
28
-
+ A search indexer, configured for image actions
29
-
+ A skillset with the new custom genAI prompt skill
30
-
+ A search index with fields to receive the verbalized text output, plus output field mappings in the indexer that establish association
25
+
+ A [supported data source](search-indexer-overview.md#supported-data-sources). We recommend Azure Storage.
26
+
+ Files or blobs containing images.
27
+
+ Read access to the supported data source. This article uses key-based authentication, but indexers can also connect using the search service identity and Microsoft Entra ID authentication. For role-based access control, assign roles on the data source to allow read access by the service identity. If you're testing on a local development machine, make sure you also have read access on the supported data source.
28
+
+ A [search indexer](search-how-to-create-indexers.md), configured for image actions.
29
+
+ A skillset with the new custom genAI prompt skill.
30
+
+ A search index with fields to receive the verbalized text output, plus output field mappings in the indexer that establish association.
31
31
32
32
Optionally, you can define projections to accept image-analyzed output into a [knowledge store](knowledge-store-concept-intro.md) for data mining scenarios.
33
33
34
34
<aname="get-normalized-images"></a>
35
35
36
36
## Configure indexers for image processing
37
37
38
-
After the source files are set up, enable image normalization by setting the `imageAction` parameter in indexer configuration. Image normalization helps make images more uniform for downstream processing. Image normalization includes the following operations:
38
+
After the source files are set up, enable image normalization by setting the `imageAction` parameter in the indexer configuration. Image normalization helps make images more uniform for downstream processing. Image normalization includes the following operations:
39
39
40
40
+ Large images are resized to a maximum height and width to make them uniform.
41
41
+ For images that have metadata that specifies orientation, image rotation is adjusted for vertical loading.
Copy file name to clipboardExpand all lines: articles/search/cognitive-search-concept-image-scenarios.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,6 +16,7 @@ ms.custom:
16
16
17
17
Images often contain useful information that's relevant in search scenarios. You can [vectorize images](search-get-started-portal-image-search.md) to represent visual content in your search index. Or, you can use [AI enrichment and skillsets](cognitive-search-concept-intro.md) to create and extract searchable *text* from images, including:
18
18
19
+
+[GenAI Prompt](cognitive-search-skill-genai-prompt.md) to pass a prompt to a chat completion skill, requesting a description of image content.
19
20
+[OCR](cognitive-search-skill-ocr.md) for optical character recognition of text and digits
20
21
+[Image Analysis](cognitive-search-skill-image-analysis.md) that describes images through visual features
21
22
+[Custom skills](#passing-images-to-custom-skills) to invoke any external image processing that you want to provide
Copy file name to clipboardExpand all lines: articles/search/cognitive-search-skill-document-extraction.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,9 +19,9 @@ The **Document Extraction** skill extracts content from a file within the enrich
19
19
20
20
For [vector](vector-search-overview.md) and [multimodal search](multimodal-search-overview.md), Document Extraction combined with the [Text Split skill](cognitive-search-skill-textsplit.md) is more affordable than other [data chunking approaches](vector-search-how-to-chunk-documents.md). The following tutorials demonstrate skill usage for different scenarios:
21
21
22
-
+[Tutorial: Index mixed content using multimodal embeddings and the Document Extraction skill](tutorial-document-extraction-multimodal-embeddings.md)
22
+
+[Tutorial: Vectorize images and text](tutorial-document-extraction-multimodal-embeddings.md)
23
23
24
-
+[Tutorial: Index mixed content using image verbalizations and the Document Extraction skill](tutorial-document-extraction-image-verbalization.md)
24
+
+[Tutorial: Verbalize images using generative AI](tutorial-document-extraction-image-verbalization.md)
25
25
26
26
> [!NOTE]
27
27
> This skill isn't bound to Azure AI services and has no Azure AI services key requirement.
Copy file name to clipboardExpand all lines: articles/search/cognitive-search-skill-document-intelligence-layout.md
+6-7Lines changed: 6 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,16 +22,15 @@ The **Document Layout** skill analyzes a document to detect structure and charac
22
22
23
23
This article is the reference documentation for the Document Layout skill. For usage information, see [How to chunk and vectorize by document layout](search-how-to-semantic-chunking.md).
24
24
25
-
It's common to use this skill on content such as PDFs that have structure and images. The following tutorials demonstrate several scenarios:
25
+
This skill uses the [Document Intelligence layout model](/azure/ai-services/document-intelligence/concept-layout) provided in [Azure AI Document Intelligence](/azure/ai-services/document-intelligence/overview).
26
26
27
-
+[Tutorial: Index mixed content using image verbalizations and the Document Layout skill](tutorial-document-layout-image-verbalization.md)
27
+
This skill is bound to a [billable Azure AI multi-service resource](cognitive-search-attach-cognitive-services.md) for transactions that exceed 20 documents per indexer per day. Execution of built-in skills is charged at the existing [Azure AI services Standard price](https://azure.microsoft.com/pricing/details/cognitive-services/).
28
28
29
-
+[Tutorial: Index mixed content using multimodal embeddings and the Document Layout skill](tutorial-document-layout-multimodal-embeddings.md)
30
-
31
-
> [!NOTE]
32
-
> This skill uses the [Document Intelligence layout model](/azure/ai-services/document-intelligence/concept-layout) provided in [Azure AI Document Intelligence](/azure/ai-services/document-intelligence/overview).
29
+
> [!TIP]
30
+
> It's common to use this skill on content such as PDFs that have structure and images. The following tutorials demonstrate image verbalization with two different data chunking techniques:
33
31
>
34
-
> This skill is bound to a [billable Azure AI multi-service resource](cognitive-search-attach-cognitive-services.md) for transactions that exceed 20 documents per indexer per day. Execution of built-in skills is charged at the existing [Azure AI services Standard price](https://azure.microsoft.com/pricing/details/cognitive-services/).
32
+
> -[Tutorial: Verbalize images from a structured document layout](tutorial-document-layout-image-verbalization.md)
33
+
> -[Tutorial: Vectorize from a structured document layout](tutorial-document-layout-multimodal-embeddings.md)
The **GenAI (Generative AI) Prompt** skill executes a *chat completion* request against a Large Language Model (LLM) deployed in Azure AI Foundry or Azure OpenAI in Azure AI Foundry Models.
18
+
The **GenAI (Generative AI) Prompt** skill executes a *chat completion* request against a Large Language Model (LLM) deployed in Azure AI Foundry or Azure OpenAI in Azure AI Foundry Models. Use this capability to create new information that can be indexed and stored as searchable content.
19
19
20
-
Use this capability to create new information that can be indexed and stored as searchable content. Examples include verbalize images, summarize larger passages, simplify complex content, or any other task that an LLM can perform. The skill supports text, image, and multimodal content such as a PDF that contains text and images. It's common to use this skill combined with a data chunking skill. The following tutorials demonstrate the image verbalization scenarios with two different data chunking techniques:
20
+
Here are some examples of how the GenAI prompt skill can help you create content:
21
21
22
-
-[Tutorial: Index mixed content using image verbalizations and the Document Layout skill](tutorial-document-layout-image-verbalization.md)
22
+
- Verbalize images
23
+
- Summarize large passages of text
24
+
- Simplify complex content
25
+
- Perform any other task that you can articulate in a prompt
23
26
24
-
-[Tutorial: Index mixed content using image verbalizations and the Document Extraction skill](tutorial-document-extraction-image-verbalization.md)
27
+
The GenAI Prompt skill is available in the [2025-05-01-preview REST API](/rest/api/searchservice/skillsets/create?view=rest-searchservice-2025-05-01-preview&preserve-view=true) only. The skill supports text, image, and multimodal content such as a PDF that contains text and images.
25
28
26
-
The GenAI Prompt skill is available in the [2025-05-01-preview REST API](/rest/api/searchservice/skillsets/create?view=rest-searchservice-2025-05-01-preview&preserve-view=true) only.
29
+
> [!TIP]
30
+
> It's common to use this skill combined with a data chunking skill. The following tutorials demonstrate image verbalization with two different data chunking techniques:
31
+
>
32
+
> -[Tutorial: Verbalize images using generative AI](tutorial-document-extraction-image-verbalization.md)
33
+
> -[Tutorial: Verbalize images from a structured document layout](tutorial-document-layout-image-verbalization.md)
Copy file name to clipboardExpand all lines: articles/search/knowledge-store-projection-example-long.md
+11-8Lines changed: 11 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,22 +7,22 @@ manager: nitinme
7
7
author: HeidiSteen
8
8
ms.author: heidist
9
9
ms.service: azure-ai-search
10
-
ms.topic: conceptual
11
-
ms.date: 06/17/2025
10
+
ms.topic: concept-article
11
+
ms.date: 07/28/2025
12
12
ms.custom:
13
13
- ignite-2023
14
14
- sfi-ropc-nochange
15
15
---
16
16
17
-
# Detailed example of shapes and projections in a knowledge store
17
+
# Example of shapes and projections in a knowledge store
18
18
19
-
This article provides a detailed example that supplements [high-level concepts](knowledge-store-projection-overview.md) and [syntax-based articles](knowledge-store-projections-examples.md) by walking you through the shaping and projection steps required for fully expressing the output of a rich skillset in a [knowledge store](knowledge-store-concept-intro.md).
19
+
This article provides a detailed example that supplements [high-level concepts](knowledge-store-projection-overview.md) and [syntax-based articles](knowledge-store-projections-examples.md) by walking you through the shaping and projection steps required for fully expressing the output of a rich skillset in a [knowledge store](knowledge-store-concept-intro.md) in Azure Storage.
20
20
21
-
If your application requirements call for multiple skills and projections, this example can give you a better idea of how shapes and projections intersect.
21
+
If your application requirements call for multiple skills and projections, this example can give you a better idea of how shapes and projections interact.
22
22
23
23
## Set up sample data
24
24
25
-
Sample documents aren't included with the Projections collection, but the [AI enrichment demo data files](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/ai-enrichment-mixed-media) contain text and images that work with the projections described in this example.
25
+
Sample documents aren't included with the Projections collection, but the [AI enrichment demo data files](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/ai-enrichment-mixed-media) contain text and images that work with the projections described in this example. If you use this sample data, you can skip step that [attaches an Azure AI multi-service account](cognitive-search-attach-cognitive-services.md) because you stay under the daily indexer limit for free enrichments.
26
26
27
27
Create a blob container in Azure Storage and upload all 14 items.
28
28
@@ -39,7 +39,7 @@ Pay close attention to skill outputs (targetNames). Outputs written to the enric
39
39
```json
40
40
{
41
41
"name": "projections-demo-ss",
42
-
"description": "Skillset that enriches blob data found in "merged_content". The enrichment granularity is a document.",
42
+
"description": "Skillset that enriches blob data found in the merged_content field. The enrichment granularity is a document.",
"description": "An Azure AI services resource in the same region as Search.",
185
-
"key": "<Azure AI services All-in-ONE KEY>"
185
+
"key": ""
186
186
},
187
187
"knowledgeStore": null
188
188
}
189
189
```
190
190
191
+
> [!NOTE]
192
+
> Under `"cognitiveServices"`, the key field is unspecified because the indexer can use an Azure AI multi-service account in the same region as your search service and process up to 20 transactions daily at no charge. The sample data for this example stays under the 20 transaction limit.
193
+
191
194
## Example Shaper skill
192
195
193
196
A [Shaper skill](cognitive-search-skill-shaper.md) is a utility for working with existing enriched content instead of creating new enriched content. Adding a Shaper to a skillset lets you create a custom shape that you can project into table or blob storage. Without a custom shape, projections are limited to referencing a single node (one projection per output), which isn't suitable for tables. Creating a custom shape aggregates various elements into a new logical whole that can be projected as a single table, or sliced and distributed across a collection of tables.
Copy file name to clipboardExpand all lines: articles/search/multimodal-search-overview.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -116,8 +116,8 @@ To help you get started with multimodal search in Azure AI Search, here's a coll
116
116
| Content | Description |
117
117
|--|--|
118
118
|[Quickstart: Multimodal search in the Azure portal](search-get-started-portal-image-search.md)| Create and test a multimodal index in the Azure portal using the wizard and Search Explorer. |
119
-
|[Tutorial: Image verbalization and Document Extraction skill](tutorial-document-extraction-image-verbalization.md)| Extract text and images, verbalize diagrams, and embed the resulting descriptions and text into a searchable index. |
120
-
|[Tutorial: Multimodal embeddings and Document Extraction skill](tutorial-document-extraction-multimodal-embeddings.md)| Use a vision-text model to embed both text and images directly, enabling visual-similarity search over scanned PDFs. |
121
-
|[Tutorial: Image verbalization and Document Layout skill](tutorial-document-layout-image-verbalization.md)| Apply layout-aware chunking and diagram verbalization, capture location metadata, and store cropped images for precise citations and page highlights. |
122
-
|[Tutorial: Multimodal embeddings and Document Layout skill](tutorial-document-layout-multimodal-embeddings.md)| Combine layout-aware chunking with unified embeddings for hybrid semantic and keyword search that returns exact hit locations. |
119
+
|[Tutorial: Verbalize images using generative AI](tutorial-document-extraction-image-verbalization.md)| Extract text and images, verbalize diagrams, and embed the resulting descriptions and text into a searchable index. |
120
+
|[Tutorial: Vectorize images and text](tutorial-document-extraction-multimodal-embeddings.md)| Use a vision-text model to embed both text and images directly, enabling visual-similarity search over scanned PDFs. |
121
+
|[Tutorial: Verbalize images from a structured document layout](tutorial-document-layout-image-verbalization.md)| Apply layout-aware chunking and diagram verbalization, capture location metadata, and store cropped images for precise citations and page highlights. |
122
+
|[Tutorial: Vectorize from a structured document layout](tutorial-document-layout-multimodal-embeddings.md)| Combine layout-aware chunking with unified embeddings for hybrid semantic and keyword search that returns exact hit locations. |
123
123
|[Sample app: Multimodal RAG GitHub repository](https://aka.ms/azs-multimodal-sample-app-repo)| An end-to-end, code-ready RAG application with multimodal capabilities that surfaces both text snippets and image annotations. Ideal for jump-starting enterprise copilots. |
0 commit comments