Skip to content

Commit 190f601

Browse files
JamesJBarnettJamesJBarnett
authored
Merge pull request #632 from HeidiSteen/heidist-rag
[azure search] V1 relevance rag tutorial
2 parents 06f8fde + 2170998 commit 190f601

5 files changed

+238
-6
lines changed

‎articles/search/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,8 @@ items:
104104
href: tutorial-rag-build-solution-pipeline.md
105105
- name: Search and generate answers
106106
href: tutorial-rag-build-solution-query.md
107+
- name: Maximize relevance
108+
href: tutorial-rag-build-solution-maximize-relevance.md
107109
- name: Skills tutorials
108110
items:
109111
- name: C#

‎articles/search/tutorial-rag-build-solution-index-schema.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: 'RAG Tutorial: Design an index'
2+
title: 'RAG tutorial: Design an index'
33
titleSuffix: Azure AI Search
44
description: Design an index for RAG patterns in Azure AI Search.
55

‎articles/search/tutorial-rag-build-solution-maximize-relevance.md

Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
1+
---
2+
title: 'RAG tutorial: Tune relevance'
3+
titleSuffix: Azure AI Search
4+
description: Learn how to use the relevance tuning capabilities to return high quality results for generative search.
5+
6+
manager: nitinme
7+
author: HeidiSteen
8+
ms.author: heidist
9+
ms.service: cognitive-search
10+
ms.topic: tutorial
11+
ms.date: 10/05/2024
12+
13+
---
14+
15+
# Tutorial: Maximize relevance (RAG in Azure AI Search)
16+
17+
In this tutorial, learn how to improve the relevance of search results used in RAG solutions. Relevance tuning can be an important factor in delivering a RAG solution that meets user expectations. In Azure AI Search, relevance tuning includes L2 semantic ranking and scoring profiles.
18+
19+
To implement these capabilities, you revisit the index schema to add configurations for semantic ranking and scoring profiles. You then rerun the queries using the new constructs.
20+
21+
In this tutorial, you modify the existing search index and queries to use:
22+
23+
> [!div class="checklist"]
24+
> - L2 semantic ranking
25+
> - Scoring profile for document boosting
26+
27+
This tutorial updates the search index created by the [indexing pipeline](tutorial-rag-build-solution-pipeline.md). Updates don't affect the existing content, so no rebuild is necessary and you don't need to rerun the indexer.
28+
29+
> [!NOTE]
30+
> There are more relevance features in preview, including vector query weighting and setting minimum thresholds, but we omit them from this tutorial becaues they aren't yet available in the Azure SDK for Python.
31+
32+
## Prerequisites
33+
34+
- [Visual Studio Code](https://code.visualstudio.com/download) with the [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python) and the [Jupyter package](https://pypi.org/project/jupyter/).
35+
36+
- [Azure AI Search](search-create-service-portal.md), Basic tier or higher for managed identity and semantic ranking, in the same region as Azure OpenAI and Azure AI Services.
37+
38+
- [Azure OpenAI](/azure/ai-services/openai/how-to/create-resource), with a deployment of text-embedding-002 and gpt-35-turbo, in the same region as Azure AI Search.
39+
40+
## Download the sample
41+
42+
The [sample notebook](https://github.com/Azure-Samples/azure-search-python-samples/blob/main/Tutorial-RAG/Tutorial-rag.ipynb) includes an updated index and query request.
43+
44+
## Update the index for semantic ranking and scoring profiles
45+
46+
In a previous tutorial, you [designed an index schema](tutorial-rag-build-solution-index-schema.md) for RAG workloads. We purposely omitted relevance enhancements from that schema so that you could focus on the fundamentals. Deferring relevance to a separate exercise also gives you a before-and-after comparison of the quality of search results after the updates are made.
47+
48+
1. Update the import statements to include classes for semantic ranking and scoring profiles.
49+
50+
```python
51+
from azure.identity import DefaultAzureCredential
52+
from azure.identity import get_bearer_token_provider
53+
from azure.search.documents.indexes import SearchIndexClient
54+
from azure.search.documents.indexes.models import (
55+
SearchField,
56+
SearchFieldDataType,
57+
VectorSearch,
58+
HnswAlgorithmConfiguration,
59+
VectorSearchProfile,
60+
AzureOpenAIVectorizer,
61+
AzureOpenAIVectorizerParameters,
62+
SearchIndex,
63+
SemanticConfiguration,
64+
SemanticPrioritizedFields,
65+
SemanticField,
66+
SemanticSearch,
67+
ScoringProfile,
68+
TagScoringFunction,
69+
TagScoringParameters
70+
)
71+
```
72+
73+
1. Add the following semantic configuration to the search index. This example can be found in the update schema step in the notebook.
74+
75+
```python
76+
# New semantic configuration
77+
semantic_config = SemanticConfiguration(
78+
name="my-semantic-config",
79+
prioritized_fields=SemanticPrioritizedFields(
80+
title_field=SemanticField(field_name="title"),
81+
keywords_fields=[SemanticField(field_name="locations")],
82+
content_fields=[SemanticField(field_name="chunk")]
83+
)
84+
)
85+
86+
# Create the semantic settings with the configuration
87+
semantic_search = SemanticSearch(configurations=[semantic_config])
88+
```
89+
90+
A semantic configuration has a name and a prioritized list of fields to help optimize the inputs to semantic ranker. For more information, see [Configure semantic ranking](/azure/search/semantic-how-to-configure).
91+
92+
1. Next, add a scoring profile definition. As with semantic configuration, a scoring profile can be added to an index schema at any time. This example is also in the update schema step in the notebook, following the semantic configuration.
93+
94+
```python
95+
# New scoring profile
96+
scoring_profiles = [
97+
ScoringProfile(
98+
name="my-scoring-profile",
99+
functions=[
100+
TagScoringFunction(
101+
field_name="locations",
102+
boost=5.0,
103+
parameters=TagScoringParameters(
104+
tags_parameter="tags",
105+
),
106+
)
107+
]
108+
)
109+
]
110+
```
111+
112+
This profile uses the tag function which boosts the scores of documents where a match was found in the locations field. Recall that the search index has a vector field, and multiple nonvector fields for title, chunks, and locations. The locations field is a string collection, and string collections can be boosted using the tags function in a scoring profile. For more information, see [Add a scoring profile](index-add-scoring-profiles.md) and [Enhancing Search Relevance with Document Boosting (blog post)](https://farzzy.hashnode.dev/enhance-azure-ai-search-document-boosting).
113+
114+
1. Update the index definition on the search service.
115+
116+
```python
117+
# Update the search index with the semantic configuration
118+
index = SearchIndex(name=index_name, fields=fields, vector_search=vector_search, semantic_search=semantic_search, scoring_profiles=scoring_profiles)
119+
result = index_client.create_or_update_index(index)
120+
print(f"{result.name} updated")
121+
```
122+
123+
## Update queries for semantic ranking and scoring profiles
124+
125+
In a previous tutorial, you [ran queries](tutorial-rag-build-solution-query.md) that execute on the search engine, passing the response and other information to an LLM for chat completion.
126+
127+
This example modifies the query request to include the semantic configuration and scoring profile.
128+
129+
```python
130+
# Import libraries
131+
from azure.search.documents import SearchClient
132+
from openai import AzureOpenAI
133+
134+
token_provider = get_bearer_token_provider(credential, "https://cognitiveservices.azure.com/.default")
135+
openai_client = AzureOpenAI(
136+
api_version="2024-06-01",
137+
azure_endpoint=AZURE_OPENAI_ACCOUNT,
138+
azure_ad_token_provider=token_provider
139+
)
140+
141+
deployment_name = "gpt-35-turbo"
142+
143+
search_client = SearchClient(
144+
endpoint=AZURE_SEARCH_SERVICE,
145+
index_name=index_name,
146+
credential=credential
147+
)
148+
149+
# Prompt is unchanged in this update
150+
GROUNDED_PROMPT="""
151+
You are an AI assistant that helps users learn from the information found in the source material.
152+
Answer the query using only the sources provided below.
153+
Use bullets if the answer has multiple points.
154+
If the answer is longer than 3 sentences, provide a summary.
155+
Answer ONLY with the facts listed in the list of sources below.
156+
If there isn't enough information below, say you don't know.
157+
Do not generate answers that don't use the sources below.
158+
Query: {query}
159+
Sources:\n{sources}
160+
"""
161+
162+
# Queries are unchanged in this update
163+
query="how much of earth is covered by water"
164+
vector_query = VectorizableTextQuery(text=query, k_nearest_neighbors=1, fields="text_vector", exhaustive=True)
165+
166+
# Add query_type semantic and semantic_configuration_name
167+
# Add scoring_profile and scoring_parameters
168+
search_results = search_client.search(
169+
query_type="semantic",
170+
semantic_configuration_name="my-semantic-config",
171+
scoring_profile="my-scoring-profile",
172+
scoring_parameters=["tags-ocean, 'sea surface', seas, surface"],
173+
search_text=query,
174+
vector_queries= [vector_query],
175+
select="title, chunk, locations",
176+
top=5,
177+
)
178+
sources_formatted = "\n".join([f'{document["title"]}:{document["chunk"]}:{document["locations"]}' for document in search_results])
179+
180+
response = openai_client.chat.completions.create(
181+
messages=[
182+
{
183+
"role": "user",
184+
"content": GROUNDED_PROMPT.format(query=query, sources=sources_formatted)
185+
}
186+
],
187+
model=deployment_name
188+
)
189+
190+
print(response.choices[0].message.content)
191+
```
192+
193+
<!-- ## Update queries for minimum thresholds ** NOT AVAILABLE IN PYTHON SDK
194+
195+
Keyword search only returns results if there's match found in the index, up to a maximum of 50 results by default. In contrast, vector search returns `k`-results every time, even if the matching vectors aren't a close match.
196+
197+
In the vector query portion of the request, add a threshold object and set a minimum value for including vector matches in the results.
198+
199+
Vector scores range from 0.333 to 1.00. For more information, see [Set thresholds to exclude low-scoring results](vector-search-how-to-query.md#set-thresholds-to-exclude-low-scoring-results-preview) and [Scores in a vector search results](vector-search-ranking.md#scores-in-a-vector-search-results).
200+
201+
```python
202+
# Update the vector_query to include a minimum threshold.
203+
query="how much of earth is covered by water"
204+
vector_query = VectorizableTextQuery(text=query, k_nearest_neighbors=1, fields="text_vector", threshold.kind="vectorSImiliarty", threshold.value=0.8, exhaustive=True) -->
205+
206+
<!-- ## Update queries for vector weighting
207+
208+
<!-- Using preview features, you can unpack a hybrid search score to review the individual component scores. Based on that information, you can set minimum thresholds to exclude any match that falls below it.
209+
210+
Semantic ranking and scoring profiles operate on nonvector content, but you can tune the vector portion of a hybrid query to amplify or diminish its importance based on how much value it adds to the results. For example, if you run keyword search and vector search independently and find that one of them is outperforming the other, you can adjust the weight on the vector side to higher or lower. This approach gives you more control over query processing.
211+
-->
212+
213+
<!-- Key points:
214+
215+
- How to measure relevance (?) to determine if changes are improving results
216+
- Try different algorithms (HNSW vs eKnn)
217+
- Change query structure (hybrid with vector/non over same content (double-down), hybrid over multiple fields)
218+
- semantic ranking
219+
- scoring profiles
220+
- thresholds for minimum score
221+
- set weights
222+
- filters
223+
- analyzers and normalizers
224+
- advanced query formats (regular expressions, fuzzy search) -->
225+
226+
<!-- ## Next step
227+
228+
> [!div class="nextstepaction"]
229+
> [Reduce vector storage and costs](tutorial-rag-build-solution-minimize-storage.md)
230+
-->

‎articles/search/tutorial-rag-build-solution-pipeline.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: 'RAG Tutorial: Build an indexing pipeline'
2+
title: 'RAG tutorial: Build an indexing pipeline'
33
titleSuffix: Azure AI Search
44
description: Create an indexer-driven pipeline that loads, chunks, embeds, and ingests content for RAG solutions on Azure AI Search.
55

‎articles/search/tutorial-rag-build-solution-query.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: 'RAG Tutorial: Search using an LLM'
2+
title: 'RAG tutorial: Search using an LLM'
33
titleSuffix: Azure AI Search
44
description: Learn how to build queries and engineer prompts for LLM-enabled search on Azure AI Search. Queries used in generative search provide the inputs to an LLM chat engine.
55

@@ -132,7 +132,7 @@ It's expected for LLMs to return different answers, even if the prompt and queri
132132
133133
## Add a filter
134134

135-
Recall that you created a `locations` field using applied AI, populated with places recognized by the Entity Recognition skill. The field definition for locations includes the `filterable` attribute. Let's repeat the previous request, but this time adding a filter that selects on the term *ice* in the locations field. A filter introduces inclusion or exclusion criteria. The search engine is still doing a vector search on `"how much of earth is covered by water"`, but it's now excluding matches that don't include *ice*. For more information about filtering on string collections and on vector queries, see [text filter fundamentals](search-filters.md#text-filter-fundamentals),[Understand collection filters](search-query-understand-collection-filters.md), and [Add filters to a vector query](vector-search-filters.md).
135+
Recall that you created a `locations` field using applied AI, populated with places recognized by the Entity Recognition skill. The field definition for locations includes the `filterable` attribute. Let's repeat the previous request, but this time adding a filter that selects on the term *ice* in the locations field. A filter introduces inclusion or exclusion criteria. The search engine is still doing a vector search on `"how much of earth is covered by water"`, but it's now excluding matches that don't include *ice*. For more information about filtering on string collections and on vector queries, see [text filter fundamentals](search-filters.md#text-filter-fundamentals), [Understand collection filters](search-query-understand-collection-filters.md), and [Add filters to a vector query](vector-search-filters.md).
136136

137137
Replace the search_results definition with the following example that includes a filter:
138138

@@ -230,7 +230,7 @@ Tasks:
230230
- H2 Query using vectors and text-to-vector conversion at query time (not sure what the code looks like for this)
231231
- H2 Query parent-child two indexes (unclear how to do this, Carey said query on child, do a lookup query on parent) -->
232232

233-
<!-- ## Next step
233+
## Next step
234234

235235
> [!div class="nextstepaction"]
236-
> [Maximize relevance](tutorial-rag-build-solution-maximize-relevance.md) -->
236+
> [Maximize relevance](tutorial-rag-build-solution-maximize-relevance.md)

0 commit comments

Comments
 (0)