You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In Azure AI Search, *agentic retrieval* is a new query pipeline designed for complex questions posed by users or agents in chat and copilot apps. It uses a large language model (LLM) to break down a question into smaller subqueries, often using chat history for context. These subqueries run in parallel, each searching for the most relevant content in your index. The results are ranked for semantic relevance, combined, and sent back to your LLM to help generate accurate answers using your proprietary content.
21
+
In Azure AI Search, *agentic retrieval* is a new multi-query pipeline designed for complex questions posed by users or agents in chat and copilot apps. It's characterized by:
22
+
23
+
+ Using a large language model (LLM) to break down a complex query into smaller subqueries. You can specify whether to include contextual chat history as a subquery.
24
+
25
+
+ Running the subqueries run in parallel, each one searching for the most relevant content in your index.
26
+
27
+
+ Combining the results into a long unified string that's sent back to your LLM for response generation, using your proprietary content to formulate the answer.
22
28
23
29
Programmatically, agentic retrieval is supported through a new Knowledge Agents object in the 2025-05-01-preview data plane REST API and in Azure SDK prerelease packages that provide the feature. A knowledge agent's retrieval response is designed for downstream consumption by other agents and chat apps.
24
30
25
31
## Why use agentic retrieval
26
32
27
33
You should use agentic retrieval when you want to provide agents and apps with the most relevant content for answering harder questions, leveraging chat context and your proprietary content.
28
34
29
-
The *agentic* aspect is a reasoning step in query planning processing that's performed by a supported large language model (LLM) that you provide. The LLM analyzes the entire chat thread to identify the underlying information need. Instead of a single, catch-all query, the model breaks down compound questions into focused subqueries based on: user questions, chat history, and parameters on the request. The subqueries target your indexed documents (plain text and vectors) in Azure AI Search.This hybrid approach ensures you surface both keyword matches and semantic similarities at once, dramatically improving recall.
35
+
The *agentic* aspect is a reasoning step in query planning processing that's performed by a supported large language model (LLM) that you provide. The LLM analyzes the entire chat thread to identify the underlying information need. Instead of a single, catch-all query, the LLM breaks down compound questions into focused subqueries based on: user questions, chat history, and parameters on the request. The subqueries target your indexed documents (plain text and vectors) in Azure AI Search.This hybrid approach ensures you surface both keyword matches and semantic similarities at once, dramatically improving recall.
30
36
31
37
The *retrieval* component is the ability to run subqueries simultaneously, merge results, semantically rank results, and return a three-part response that includes grounding data for the next conversation turn, reference data so that you can inspect the source content, and an activity plan that shows query execution steps.
32
38
@@ -48,7 +54,7 @@ Agentic retrieval invokes the entire query processing pipeline multiple times fo
48
54
> [!NOTE]
49
55
> Including an LLM in query planning adds latency to a query pipeline. You can mitigate the effects by using faster models, such as gpt-4o-mini, and summarizing the message threads. Nonetheless, you should expect longer query times with this pipeline.
50
56
51
-
## Agentic retrieval architecture
57
+
## Architecture and components
52
58
53
59
Agentic retrieval is designed for a conversational search experience that includes an LLM. An important part of agentic retrieval is how the LLM breaks down an initial query into subqueries, which are more effective at locating the best matches in your index.
54
60
@@ -78,6 +84,32 @@ Agentic retrieval has these processes:
78
84
79
85
Your search index determines query execution and any optimizations that occur during query execution. This includes your semantic configuration, as well as optional scoring profiles, synonym maps, analyzers, and normalizers (if you add filters).
80
86
87
+
## How to get started
88
+
89
+
You must use the preview REST APIs or a prerelease Azure SDK package that provides the functionality. At this time, there's no Azure portal or Azure AI Foundry portal support.
90
+
91
+
Choose any of these options for your next step.
92
+
93
+
<!-- + Watch this demo. -->
94
+
+[Quickstart article: Run agentic retrieval in Azure AI Search](search-get-started-agentic-retrieval.md). Learn the basic workflow using sample data and a prepared index and queries.
+[End-to-end with Azure AI Search and Azure AI Agent Service](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/agentic-retrieval-pipeline-example)
102
+
103
+
+ How-to guides for a focused look at development tasks:
104
+
105
+
+[Create a knowledge agent](search-agentic-retrieval-how-to-create.md)
106
+
+[Use a knowledge agent to retrieve data](search-agentic-retrieval-how-to-retrieve.md)
107
+
+[Build an agent-to-agent retrieval solution](search-agentic-retrieval-how-to-pipeline.md).
108
+
109
+
+ REST API reference, [Knowledge Agents](/rest/api/searchservice/knowledge-agents?view=rest-searchservice-2025-05-01-preview&preserve-view=true) and [Knowledge Retrieval](/rest/api/searchservice/knowledge-retrieval/retrieve?view=rest-searchservice-2025-05-01-preview&preserve-view=true).
110
+
111
+
+[Azure OpenAI Demo](https://github.com/Azure-Samples/azure-search-openai-demo), updated to use agentic retrieval.
112
+
81
113
## Availability and pricing
82
114
83
115
Agentic retrieval is available in [all regions that provide semantic ranker](search-region-support.md), on all tiers except the free tier.
@@ -140,32 +172,6 @@ To estimate the semantic ranking costs associated with agentic retrieval, start
140
172
141
173
Putting it all together, you'd pay about $3.30 for semantic ranking in Azure AI Search, 60 cents for input tokens in Azure OpenAI, and 42 cents for output tokens in Azure OpenAI, for $1.02 for query planning total. The combined cost for the full execution is $4.32.
142
174
143
-
## How to get started
144
-
145
-
You must use the preview REST APIs or a prerelease Azure SDK package that provides the functionality. At this time, there's no Azure portal or Azure AI Foundry portal support.
146
-
147
-
Choose any of these options for your next step.
148
-
149
-
<!-- + Watch this demo. -->
150
-
+[Quickstart article: Run agentic retrieval in Azure AI Search](search-get-started-agentic-retrieval.md). Learn the basic workflow using sample data and a prepared index and queries.
+[End-to-end with Azure AI Search and Azure AI Agent Service](https://github.com/Azure-Samples/azure-search-python-samples/tree/main/agentic-retrieval-pipeline-example)
158
-
159
-
+ How-to guides for a focused look at development tasks:
160
-
161
-
+[Create a knowledge agent](search-agentic-retrieval-how-to-create.md)
162
-
+[Use a knowledge agent to retrieve data](search-agentic-retrieval-how-to-retrieve.md)
163
-
+[Build an agent-to-agent retrieval solution](search-agentic-retrieval-how-to-pipeline.md).
164
-
165
-
+ REST API reference, [Knowledge Agents](/rest/api/searchservice/knowledge-agents?view=rest-searchservice-2025-05-01-preview&preserve-view=true) and [Knowledge Retrieval](/rest/api/searchservice/knowledge-retrieval/retrieve?view=rest-searchservice-2025-05-01-preview&preserve-view=true).
166
-
167
-
+[Azure OpenAI Demo](https://github.com/Azure-Samples/azure-search-openai-demo), updated to use agentic retrieval.
168
-
169
175
<!--
170
176
•Query Pipeline Recap: The query pipeline includes stages: Query Preprocessing (Query Rewriting, Vectorization, Text analysis), Ranking (Vector Search, Keyword Search, Fusion, Semantic Ranking), and Synthesis (Results for LLM, Extractive Answers, Contextualized Captions).
0 commit comments