Skip to content

Commit 44194f2

Browse files
HeidiSteenHeidiSteen
committed
Revisions per testing semantic scoring
1 parent 01331c4 commit 44194f2

File tree

4 files changed

+77
-52
lines changed

4 files changed

+77
-52
lines changed

‎articles/search/hybrid-search-ranking.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ RRF is used anytime there's more than one query execution. The following example
4949

5050
Whenever results are ranked, **`@search.score`** property contains the value used to order the results. Scores are generated by ranking algorithms that vary for each method. Each algorithm has its own range and magnitude.
5151

52-
The following chart identifies the scoring property returned on each match, algorithm, and range of scores for each relevance ranking algorithm.
52+
The following chart identifies the scoring property returned on each match, algorithm, and range of scores for each relevance ranking algorithm. For more information and a diagram of the scoring workflow, see [Relevance in Azure AI Search](search-relevance-overview.md).
5353

5454
| Search method | Parameter | Scoring algorithm | Range |
5555
|---------------|-----------|-------------------|-------|

‎articles/search/index-add-scoring-profiles.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ You can use scoring profiles in [keyword search](search-lucene-query-architectur
3535

3636
You can have up to 100 scoring profiles within an index (see [service Limits](search-limits-quotas-capacity.md)), but you can only specify one profile at time in any given query.
3737

38-
You can use [semantic ranker](semantic-how-to-query-request.md) with scoring profiles. When multiple ranking or relevance features are in play, semantic ranking is the last step. [How search scoring works](search-relevance-overview.md#diagram-of-ranking-algorithms) provides an illustration.
38+
You can use [semantic ranker](semantic-how-to-query-request.md) with scoring profiles. Currently in preview, you can apply a [scoring profile after semantic ranking](semantic-how-to-enable-scoring-profiles.md). Otherwise, when multiple ranking or relevance features are in play, semantic ranking is the last step. [How search scoring works](search-relevance-overview.md#diagram-of-ranking-algorithms) provides an illustration.
3939

4040
[Extra rules](#rules-for-using-functions) apply specifically to functions.
4141

‎articles/search/search-relevance-overview.md

Lines changed: 58 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -25,27 +25,9 @@ This section describes the levels of scoring operations. For an illustration of
2525

2626
| Level | Description |
2727
|-------|-------------|
28-
| Level 1 (L1) | Initial search score. For text queries matching on tokenized strings, results are always initially ranked using the [BM25 ranking algorithm](index-similarity-and-scoring.md). For vector queries, results are ranked using either [Hierarchical Navigable Small World (HNSW) or exhaustive K-nearest neighbor (KNN)](vector-search-ranking.md). Image search or multimodal searches are based on vector queries and scored using the vector L2 ranking algorithms. |
29-
| Fused L1 | Unifies the search results from multiple queries and applies the [Reciprocal Ranking Fusion (RRF) algorithm](hybrid-search-ranking.md). RRF is used for hbrid queries that include text and vector components. RRF is also used when multiple vector queries execute in parallel. |
30-
| Level 2 (L2) | [Semantic ranker](semantic-search-overview.md) applies machine reading comprehension to textual content. Semantic ranking is a premium feature that bills for use of the semantic ranking models. It's optional for text queries and vector queries that contain text, but required for [agentic retrieval (preview)](search-agentic-retrieval-concept.md). Although agentic retrieval sends multiple queries to the query engine, the ranking algorithm for agentic retrieval is the semantic ranker. |
31-
32-
<!-- ### Level 1 (L1) ranking
33-
34-
Assuming the query engine performs a scoring operation, the initial search score varies by query type.
35-
36-
+ Text queries, which match on tokenized strings, are always initially ranked using the [BM25 ranking algorithm](index-similarity-and-scoring.md).
37-
38-
+ Vector query L1 ranking is either [Hierarchical Navigable Small World (HNSW) or exhaustive K-nearest neighbor (KNN)](vector-search-ranking.md). Image search or multimodal searches are based on vector queries and scored using the vector L2 ranking algorithms.
39-
40-
### Fused L1 ranking
41-
42-
Hybrid queries that include text and vector components are ranked using the [Reciprocal Ranking Fusion (RRF) algorithm](hybrid-search-ranking.md) that's used for merging the results of multiple queries. RRF is also used if multiple vector queries execute in parallel.
43-
44-
### Level 2 (L2) ranking
45-
46-
The L2 ranking feature in Azure AI Search is the [semantic ranker](semantic-search-overview.md) that applies machine reading comprehension to textual content. Semantic ranking is a premium feature that incurs extra charges for use of the semantic ranking models.
47-
48-
It's optional for text queries and vector queries that contain text, but required for [agentic retrieval (preview)](search-agentic-retrieval-concept.md). Although agentic retrieval sends multiple queries to the query engine, the ranking algorithm for agentic retrieval is the L2 ranker. -->
28+
| Level&nbsp;1&nbsp;(L1) | Initial search score (`@search.score`). <br>For text queries matching on tokenized strings, results are always initially ranked using the [BM25 ranking algorithm](index-similarity-and-scoring.md). <br>For vector queries, results are ranked using either [Hierarchical Navigable Small World (HNSW) or exhaustive K-nearest neighbor (KNN)](vector-search-ranking.md). Image search or multimodal searches are based on vector queries and scored using the L1 vector ranking algorithms. |
29+
| Fused&nbsp;L1 | Scoring from multiple queries using the [Reciprocal Ranking Fusion (RRF) algorithm](hybrid-search-ranking.md). RRF is used for hybrid queries that include text and vector components. RRF is also used when multiple vector queries execute in parallel. A search score from RRF is reflected in `@search.score` over a different range.|
30+
| Level&nbsp;2&nbsp;(L2) | [Semantic ranking score (`@search.reRankerScore`)](semantic-search-overview.md) applies machine reading comprehension to textual content. Semantic ranking is a premium feature that bills for use of the semantic ranking models. It's optional for text queries and vector queries that contain text, but required for [agentic retrieval (preview)](search-agentic-retrieval-concept.md). Although agentic retrieval sends multiple queries to the query engine, the ranking algorithm for agentic retrieval is the semantic ranker. |
4931

5032
## Custom boosting logic using scoring profiles
5133

@@ -87,28 +69,63 @@ The following diagram illustrates how the ranking algorithms work together.
8769
> [!NOTE]
8870
> This workflow diagram currently omits `@search.rerankerScoreBoosted` and a step for semantic ranking with boosting from a scoring profile. If you use semantic ranking with scoring profile, the scoring profile is applied after L2 ranking, and the final score is based on `@search.rerankerScoreBoosted`.
8971
90-
A query that generates the previous workflow might look like the following example. Because the latest preview REST API is used, this hybrid semantic query is scored using RRF (based on L1 scores for text and vectors), semantic ranking, and scoring profiles. The `@search.rerankerScoreBoosted` score determines the order.
72+
## Example query inclusive of all ranking algorithms
73+
74+
A query that generates the previous workflow might look like the following example. This hybrid semantic query is scored using RRF (based on L1 scores for text and vectors), and semantic ranking.
9175

9276
```http
9377
POST https://{{search-service-name}}.search.windows.net/indexes/{{index-name}}/docs/search?api-version=2025-05-01-preview
94-
Content-Type: application/json
95-
api-key: {{admin-api-key}}
78+
9679
{
97-
"queryType":"semantic",
98-
"search":"what is a hello world application",
99-
"searchFields":"field_a, field_b",
100-
"vectorQueries": [
101-
{
102-
"kind":"vector",
103-
"vector": [1.0, 2.0, 3.0],
104-
"fields": "field_c, field_d"
105-
},
106-
{
107-
"kind":"vector",
108-
"vector": [4.0, 5.0, 6.0],
109-
"fields": "field_d, field_e"
110-
}
111-
],
112-
"scoringProfile":"my_scoring_profile"
80+
"search": "cloud formation over water",
81+
"count": true,
82+
"vectorQueries": [
83+
{
84+
"kind": "text",
85+
"text": "cloud formation over water",
86+
"fields": "text_vector,image_vector"
87+
}
88+
],
89+
"queryType": "semantic",
90+
"semanticConfiguration": "my-semantic-configuration",
91+
"select": "title,chunk",
92+
"top": 5
11393
}
114-
```
94+
```
95+
96+
A response for the above query includes the original RRF `@search.core` and the `@search.rerankerScore`.
97+
98+
```json
99+
"value": [
100+
{
101+
"@search.score": 0.03177805617451668,
102+
"@search.rerankerScore": 2.6919238567352295,
103+
"chunk": "A\nT\n\nM\nO\n\nS\nP\n\nH\nE\n\nR\nE\n\nE\nA\n\nR\nT\n\nH\n\n32\n\nFraming an Iceberg\nSouth Atlantic Ocean\n\nIn June 2016, the Suomi NPP satellite captured this image of various cloud formations in the South Atlantic Ocean. Note how low \n\nstratus clouds framed a hole over iceberg A-56 as it drifted across the sea. \n\nThe exact reason for the hole in the clouds is somewhat of a mystery. It could have formed by chance, although imagery from the \n\ndays before and after this date suggest something else was at work. It could be that the relatively unobstructed path of the clouds \n\nover the ocean surface was interrupted by thermal instability created by the iceberg. In other words, if an obstacle is big enough, \n\nit can divert the low-level atmospheric flow of air around it, a phenomenon often caused by islands.",
104+
"title": "page-39.pdf",
105+
},
106+
{
107+
"@search.score": 0.030621785670518875,
108+
"@search.rerankerScore": 2.557225465774536,
109+
"chunk": "A\nT\n\nM\nO\n\nS\nP\n\nH\nE\n\nR\nE\n\nE\nA\n\nR\nT\n\nH\n\n24\n\nMaking Tracks\nPacific Ocean\n\nShips steaming across the Pacific Ocean left this cluster of bright cloud trails lingering in the atmosphere in February 2012. The \n\nnarrow clouds, known as ship tracks, form when water vapor condenses around tiny particles of pollution from ship exhaust. The \n\ncrisscrossing clouds off the coast of California stretched for many hundreds of kilometers from end to end. The narrow ends of the \n\nclouds are youngest, while the broader, wavier ends are older.\n\nSome of the pollution particles generated by ships (especially sulfates) are soluble in water and can serve as the seeds around which \n\ncloud droplets form. Clouds infused with ship exhaust have more and smaller droplets than unpolluted clouds. As a result, light \n\nhitting the ship tracks scatters in many directions, often making them appear brighter than other types of marine clouds, which are \n\nusually seeded by larger, naturally occurring particles like sea salt.",
110+
"title": "page-31.pdf",
111+
},
112+
{
113+
"@search.score": 0.013698630034923553,
114+
"@search.rerankerScore": 2.515575408935547,
115+
"chunk": "A\nT\n\nM\nO\n\nS\nP\n\nH\nE\n\nR\nE\n\nE\nA\n\nR\nT\n\nH\n\n16\n\nRiding the Waves\nMauritania\n\nYou cannot see it directly, but air masses from Africa and the Atlantic Ocean are colliding in this Landsat 8 image from August 2016. \n\nThe collision off the coast of Mauritania produces a wave structure in the atmosphere. \n\nCalled an undular bore or solitary wave, this cloud formation was created by the interaction between cool, dry air coming off the \n\ncontinent and running into warm, moist air over the ocean. The winds blowing out from the land push a wave of air ahead like a \n\nbow wave moving ahead of a boat. \n\nParts of these waves are favorable for cloud formation, while other parts are not. The dust blowing out from Africa appears to be \n\nriding these waves. Dust has been known to affect cloud growth, but it probably has little to do with the cloud pattern observed here.",
116+
"title": "page-23.pdf",
117+
},
118+
{
119+
"@search.score": 0.028949543833732605,
120+
"@search.rerankerScore": 2.4990925788879395,
121+
"chunk": "A\nT\n\nM\nO\n\nS\nP\n\nH\nE\n\nR\nE\n\nE\nA\n\nR\nT\n\nH\n\n14\n\nBering Streets\nArctic Ocean\n\nWinds from the northeast pushed sea ice southward and formed cloud streets—parallel rows of clouds—over the Bering Strait in \n\nJanuary 2010. The easternmost reaches of Russia, blanketed in snow and ice, appear in the upper left. To the east, sea ice spans \n\nthe Bering Strait. Along the southern edge of the ice, wavy tendrils of newly formed, thin sea ice predominate.\n\nThe cloud streets run in the direction of the northerly wind that helps form them. When wind blows out from a cold surface like sea \n\nice over the warmer, moister air near the open ocean, cylinders of spinning air may develop. Clouds form along the upward cycle in \n\nthe cylinders, where air is rising, and skies remain clear along the downward cycle, where air is falling. The cloud streets run toward \n\nthe southwest in this image from the Terra satellite.",
122+
"title": "page-21.pdf",
123+
},
124+
{
125+
"@search.score": 0.027637723833322525,
126+
"@search.rerankerScore": 2.4686081409454346,
127+
"chunk": "A\nT\n\nM\nO\n\nS\nP\n\nH\nE\n\nR\nE\n\nE\nA\n\nR\nT\n\nH\n\n38\n\nLofted Over Land\nMadagascar\n\nAlong the muddy Mania River, midday clouds form over the forested land but not the water. In the tropical rainforests of Madagascar, \n\nthere is ample moisture for cloud formation. Sunlight heats the land all day, warming that moist air and causing it to rise high into the \n\natmosphere until it cools and condenses into water droplets. Clouds generally form where air is ascending (over land in this case), \n\nbut not where it is descending (over the river). Landsat 8 acquired this image in January 2015.",
128+
"title": "page-45.pdf",
129+
}
130+
]
131+
```

‎articles/search/semantic-how-to-enable-scoring-profiles.md

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
title: Use Scoring Profiles with Semantic Ranking
33
titleSuffix: Azure AI Search
44
description: Learn how to combine scoring profiles with semantic ranking in Azure AI Search to optimize final document relevance.
5-
author: gmndrg
6-
ms.author: gimondra
5+
author: HeidiSteen
6+
ms.author: heidist
77
ms.service: azure-ai-search
88
ms.update-cycle: 180-days
99
ms.topic: how-to
@@ -14,23 +14,31 @@ ms.date: 07/22/2025
1414

1515
[!INCLUDE [Feature preview](./includes/previews/preview-generic.md)]
1616

17-
Integrating [scoring profiles](index-add-scoring-profiles.md) with [semantic ranker](semantic-search-overview.md) is supported in newer Azure AI Search preview REST API versions and Azure SDK preview packages. Semantic ranker adds a new field, `@search.rerankerBoostedScore`, that applies scoring profile logic on semantically ranked results. In search results that include `@search.score` from BM25, `@search.rerankerScore` from semantic ranker, and `@search.reRankerBoostedScore`, results are sorted by `@search.reRankerBoostedScore`.
17+
Integrating [scoring profiles](index-add-scoring-profiles.md) with [semantic ranker](semantic-search-overview.md) is supported in newer Azure AI Search preview REST API versions and Azure SDK preview packages. Semantic ranker adds a new response field, `@search.rerankerBoostedScore`, that applies scoring profile logic on semantically ranked results. In search results that include `@search.score` from level 1 ranking, `@search.rerankerScore` from semantic ranker, and `@search.reRankerBoostedScore`, results are sorted by `@search.reRankerBoostedScore`.
1818

19-
Before this integration, scoring profiles only influenced the initial L1 ranking phase of [BM25-ranked](index-similarity-and-scoring.md) and [RRF-ranked](hybrid-search-ranking.md) search results. However, once the semantic L2 ranker re-ranked the results, those boosts no longer had any effect. The semantic reranking process ignored scoring profiles entirely.
20-
21-
Integrating scoring profiles with semantic ranker addresses this behavior by applying scoring profiles to L2-ranked results, ensuring that the boosts are taken into account.
19+
If you're using a stable API version or an earlier preview, scoring profiles are used upstream, before the semantic ranking step. For a diagram of the scoring workflow, see [Relevance in Azure AI Search](search-relevance-overview.md).
2220

2321
## Prerequisites
2422

2523
- [Azure AI Search](search-create-service-portal.md), Basic pricing tier or higher, with [semantic ranker enabled](semantic-how-to-enable-disable.md).
2624

27-
- REST API version `2025-05-01-preview` or a prerelease Azure SDK package that provides the new APIs. For all preview features, we recommend reviewing the Azure SDK change logs for feature availability: [Python SDK change log](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/CHANGELOG.md), [.NET SDK change log](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/CHANGELOG.md), [Java SDK change log](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/search/azure-search-documents/CHANGELOG.md), [JavaScript SDK change log](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/search/search-documents/CHANGELOG.md).
25+
- [REST API version `2025-05-01-preview`](/rest/api/searchservice/operation-groups?view=rest-searchservice-2025-05-01-preview&preserve-view=true) or a prerelease Azure SDK package that provides the new APIs. Currently, there's no Azure portal (Search Explorer) support for this feature so use a REST client or an IDE.
26+
27+
For all preview features, we recommend reviewing the Azure SDK change logs to check for feature availability: [Python SDK change log](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/CHANGELOG.md), [.NET SDK change log](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/CHANGELOG.md), [Java SDK change log](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/search/azure-search-documents/CHANGELOG.md), [JavaScript SDK change log](https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/search/search-documents/CHANGELOG.md).
28+
29+
- A search index with a semantic configuration that specifies `"rankingOrder": "boostedReRankerScore"` and a scoring profile that specifies [functions](index-add-scoring-profiles.md#use-functions).
30+
31+
- A semantic query includes the scoring profile.
32+
33+
## Limitations
34+
35+
Boosting of semantically ranked results applies to scoring profile functions only. There's no boosting if the scoring profile consists of just weighted text fields.
2836

2937
## How does semantic configuration with scoring profiles work?
3038

3139
When you execute a semantic query associated with a scoring profile, a third search score, `@search.rerankerBoostedScore` value, is generated for every document in your search results. This boosted score, calculated by applying the scoring profile to the existing reranker score, doesn't have a guaranteed range (0–4) like a normal reranker score, and scores can be significantly higher than 4.
3240

33-
Starting in API version `2025-05-01-preview`, semantic results are sorted by `@search.rerankerBoostedScore` by default. If the `rankingOrder` property isn't specified, then `boostedReRankerScore` is the default value in the semantic configuration.
41+
Starting in API version `2025-05-01-preview`, semantic results are sorted by `@search.rerankerBoostedScore` by default if a scoring profile exists. If the `rankingOrder` property isn't specified, then `BoostedReRankerScore` is the default value in the semantic configuration.
3442

3543
When this capability is enabled, the scoring profile defined in your index applies during the initial ranking phase.
3644
It boosts results from:
@@ -64,7 +72,7 @@ PUT https://{service-name}.search.windows.com/indexes/{index-name}?api-version=2
6472
To opt out of sorting by semantic reranker boosted score, set the `rankingOrder` field to `reRankerScore` value in the semantic configuration.
6573

6674
```json
67-
PUT https://{service-name}.search.windows.com/indexes/{index-name}?api-version=2024-05-01-Preview
75+
PUT https://{service-name}.search.windows.com/indexes/{index-name}?api-version=2025-05-01-Preview
6876
{
6977
"semantic": {
7078
"configurations": [

0 commit comments

Comments
 (0)