maxTextRecallSize updates

Author: HeidiSteen

articles/search/hybrid-search-how-to-query.md

@@ -9,7 +9,7 @@ ms.service: cognitive-search
 ms.custom:
   - ignite-2023
 ms.topic: how-to
-ms.date: 04/23/2024
+ms.date: 06/12/2024
 ---
 
 # Create a hybrid query in Azure AI Search
@@ -18,7 +18,7 @@ ms.date: 04/23/2024
 
 In most cases, [per benchmark tests](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/azure-ai-search-outperforming-vector-search-with-hybrid/ba-p/3929167), hybrid queries with semantic ranking return the most relevant results.
 
-To define a hybrid query, use REST API [**2023-11-01**](/rest/api/searchservice/documents/search-post), [**2023-10-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2023-10-01-preview&preserve-view=true), [**2024-03-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-03-01-preview&preserve-view=true), Search Explorer in the Azure portal, or newer versions of the Azure SDKs. 
+New parameters [maxTextRecallSize and countAndFacetMode(preview)](#relevance-tuning-through-maxtextrecallsize-and-countandfacetmode-preview) give you more control over inputs into a hybrid query. You can also use [vector weighting](vector-search-how-to-query.md#vector-weighting-preview) on the vector side of a hybrid query to specify the relative weight of the vector query. This feature is particularly useful in complex queries where two or more distinct result sets need to be combined, as is the case for hybrid search. 
 
 ## Prerequisites
 
@@ -28,6 +28,15 @@ To define a hybrid query, use REST API [**2023-11-01**](/rest/api/searchservice/
 
 + (Optional) If you want text-to-vector conversion of a query string (currently in preview), [create and assign a vectorizer](vector-search-how-to-configure-vectorizer.md) to vector fields in the search index.
 
+## Choose an API or tool
+
++ [**2023-11-01**](/rest/api/searchservice/documents/search-post) stable version
++ [**2023-10-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2023-10-01-preview&preserve-view=true), adds integrated vectorization to the vector side of a hybrid query
++ [**2024-03-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-03-01-preview&preserve-view=true), adds narrow data types and scalar quantization to the vector side of a hybrid query
++ [**2024-05-01-preview**](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-05-01-preview&preserve-view=true) adds `maxTextRecallSize` and `countAndFacetMode` specifially for hybrid search
++ Search Explorer in the Azure portal (targets 2024-05-01-preview behaviors)
++ Newer stable or beta packages of the Azure SDKs (see change logs for SDK feature support)
+
 ## Run a hybrid query in Search Explorer
 
 1. In [Search Explorer](search-explorer.md), make sure the API version is **2023-10-01-preview** or later.
@@ -222,7 +231,37 @@ api-key: {{admin-api-key}}
 
 + Prefiltering is applied before query execution. If prefilter reduces the search area to 100 documents, the vector query executes over the "DescriptionVector" field for those 100 documents, returning the k=50 best matches. Those 50 matching documents then pass to RRF for merged results, and then to semantic ranker.
 
-+ Postfilter is applied after query execution. If k=50 returns 50 matches on the vector query side, then the post-filter is applied to the 50 matches, reducing results that meet filter criteria, leaving you with fewer than 50 documents to pass to semantic ranker
++ Postfilter is applied after query execution. If k=50 returns 50 matches on the vector query side, then the post-filter is applied to the 50 matches, reducing results that meet filter criteria, leaving you with fewer than 50 documents to pass to semantic ranker.
+
+## Relevance tuning through maxTextRecallSize and countAndFacetMode (preview)
+
+In a `hybridSearch` query parameter object, specify the maximum number of documents recalled through the text portion of a hybrid (text and vector) query. The default is `1000` documents. With this parameter, you can increase or decrease the number of results returned in hybrid queries. Reducing the number of text documents retrieved can significantly improve performance. 
+
+You can set `countAndFacetMode` property to report the counts for the text query (and for facets if you're using them) to the document limits set by `maxTextRecallSize`.
+
+Use [Search - POST](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-05-01-preview&preserve-view=true) or [Search - GET](/rest/api/searchservice/documents/search-get?view=rest-searchservice-2024-05-01-preview&preserve-view=true) in `2024-05-01-Preview` to specify these parameters.
+
+The following example sets `maxTextRecallSize` to 100, which limits the results from the text side of the hybrid query to just 100 documents instead of all matches (up to 1,000 document default). It also sets `countAndFacetMode` to include only those results from `maxTextRecallSize`.
+
+```http
+POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?api-version=2024-05-01-Preview 
+
+    { 
+      "vectorQueries": [ 
+        { 
+          "kind": "vector", 
+          "vector": [1.0, 2.0, 3.0], 
+          "fields": "my_vector_field", 
+          "k": 10 
+        } 
+      ], 
+      "search": "hello world", 
+      "hybridSearch": { 
+        "maxTextRecallSize": 100, 
+        "countAndFacetMode": "countRetrievableResults" 
+      } 
+    } 
+```
 
 ## Configure a query response
 

articles/search/hybrid-search-ranking.md

@@ -9,7 +9,7 @@ ms.service: cognitive-search
 ms.custom:
   - ignite-2023
 ms.topic: conceptual
-ms.date: 11/01/2023
+ms.date: 06/12/2024
 ---
 
 # Relevance scoring in hybrid search using Reciprocal Rank Fusion (RRF)

articles/search/vector-search-how-to-query.md

@@ -9,7 +9,7 @@ ms.service: cognitive-search
 ms.custom:
   - build-2024
 ms.topic: how-to
-ms.date: 05/30/2024
+ms.date: 06/12/2024
 ---
 
 # Create a vector query in Azure AI Search
@@ -22,7 +22,6 @@ In Azure AI Search, if you have a [vector index](vector-search-how-to-create-ind
 > + [Query multiple vector fields at once](#multiple-vector-fields)
 > + [Query with integrated vectorization (preview)](#query-with-integrated-vectorization-preview)
 > + [Set thresholds to exclude low-scoring results (preview)](#set-thresholds-to-exclude-low-scoring-results-preview)
-> + [Set MaxTextSizeRecall to control the number of results (preview)](#maxtextsizerecall-for-hybrid-search-preview)
 > + [Set vector weights (preview)](#vector-weighting-preview)
 
 This article uses REST for illustration. For code samples in other languages, see the [azure-search-vector-samples](https://github.com/Azure/azure-search-vector-samples) GitHub repository for end-to-end solutions that include vector queries. 
@@ -598,14 +597,19 @@ POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?
 
 Filtering occurs before [fusing results](hybrid-search-ranking.md) from different recall sets. 
 
+ <!-- This section should be in hybrid search only, but keeping a copy here to avoid broken links in a blog post. -->
 ## MaxTextSizeRecall for hybrid search (preview)
 
-Add a `hybridSearch` query parameter object to specify the maximum number of documents recalled using text queries in hybrid (text and vector) search. The default is 1,000 documents. With this parameter, you can increase or decrease the number of results returned in hybrid queries.
+In a `hybridSearch` query parameter object, specify the maximum number of documents recalled through the text portion of a hybrid (text and vector) query. The default is `1000` documents. With this parameter, you can increase or decrease the number of results returned in hybrid queries. Reducing the number of text documents retrieved can significantly improve performance. 
+
+You can set `countAndFacetMode` property to report the counts for the text query (and for facets if you're using them) to the document limits set by `maxTextRecallSize`.
+
+Use [Search - POST](/rest/api/searchservice/documents/search-post?view=rest-searchservice-2024-05-01-preview&preserve-view=true) or [Search - GET](/rest/api/searchservice/documents/search-get?view=rest-searchservice-2024-05-01-preview&preserve-view=true) in `2024-05-01-Preview` to specify these parameters.
+
+The following example sets `maxTextRecallSize` to 100, which limits the results from the text side of the hybrid query to just 100 documents instead of all matches (up to 1,000 document default). It also sets `countAndFacetMode` to include only those results from `maxTextRecallSize`.
 
 ```http
 POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?api-version=2024-05-01-Preview 
-    Content-Type: application/json 
-    api-key: [admin key] 
 
     { 
       "vectorQueries": [ 
@@ -619,30 +623,39 @@ POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?
       "search": "hello world", 
       "hybridSearch": { 
         "maxTextRecallSize": 100, 
-        "countAndFacetMode": "countAllResults" 
+        "countAndFacetMode": "countRetrievableResults" 
       } 
     } 
 ```
 
 ## Vector weighting (preview)
 
-Add a `weight` query parameter to specify the relative weight of each vector included in search operations. This feature is particularly useful in complex queries where two or more distinct result sets need to be combined, such as in hybrid search or multivector requests. 
+Add a `weight` query parameter to specify the relative weight of each vector included in search operations. This value is used when combining the results of multiple ranking lists produced by two or more vector queries in the same request, or from the vector portion of a hybrid query.
+
+The default is 1.0 and the value must be a positive number larger than zero.
 
-Weights are used when calculating the [reciprocal rank fusion](hybrid-search-ranking.md) scores of each document. The calculation is multiplier of the `weight` value against the rank score of the document within its respective result set. 
+Weights are used when calculating the [reciprocal rank fusion](hybrid-search-ranking.md) scores of each document. The calculation is multiplier of the `weight` value against the rank score of the document within its respective result set.
+
+The following example is a hybrid query with two vector query strings and one text string. Weights are assigned to the vector queries. The first query is 0.5 or half the weight, reducing its importance in the request. The second vectory query is twice as important. Text queries have no weight parameters, but you can increase or decrease their importance by setting [maxTextRecallSize](hybrid-search-how-to-query.md#ranking).
 
 ```http
 POST https://[service-name].search.windows.net/indexes/[index-name]/docs/search?api-version=2024-05-01-Preview 
-Content-Type: application/json 
-api-key: [admin key] 
 
     { 
       "vectorQueries": [ 
         { 
           "kind": "vector", 
           "vector": [1.0, 2.0, 3.0], 
-          "fields": "my_vector_field", 
+          "fields": "my_first_vector_field", 
           "k": 10, 
           "weight": 0.5 
+        },
+        { 
+          "kind": "vector", 
+          "vector": [4.0, 5.0, 6.0], 
+          "fields": "my_second_vector_field", 
+          "k": 10, 
+          "weight": 2.0
         } 
       ], 
       "search": "hello world" 

articles/search/whats-new.md

@@ -25,7 +25,7 @@ ms.custom:
 |-----------------------------|------|--------------|
 | [Higher capacity and more vector quota at every tier (same billing rate)](search-limits-quotas-capacity.md#service-limits) | Infrastructure | Partition sizes are now even larger for Standard 2 (S2), Standard 3 (S3), and Standard 3 High Density (S3 HD) for all services created after April 3, 2024. If you create a new service now, you get the larger partitions. If you created a new service between April 3 and May 17, you get the larger partitions automatically. <br><br>Storage Optimized tiers (L1 and L2) also have more capacity. L1 and L2 customers must create a new service to benefit from the higher capacity. There's no in-place upgrade at this time. <br><br>Extra capacity is now available in [more regions](search-limits-quotas-capacity.md#supported-regions-with-higher-storage-limits): Germany North​, Germany West Central​, South Africa North​, Switzerland West​, and Azure Government (Texas, Arizona, and Virginia).|
 | [OneLake integration (preview)](search-how-to-index-onelake-files.md) | Feature | New indexer for OneLake files and OneLake shortcuts. If you use Microsoft Fabric and OneLake for data access to Amazon Web Services (AWS) and Google data sources, use this indexer to import external data into a search index. This indexer is available through the Azure portal, the [2024-05-01-preview REST API](/rest/api/searchservice/data-sources/create-or-update?view=rest-searchservice-2024-05-01-preview&preserve-view=true), and Azure SDK beta packages. |
-| [Vector relevance tuning and search results customization](vector-search-how-to-query.md) | Feature | Three enhancements improve vector search relevance. <br><br>First, you can now set thresholds on vector search results to exclude low-scoring results. <br><br>Second, you can set `MaxSizeTextRecall` and `countAndFacetMode` in hybrid queries to specify the maximum number of documents that can be recalled using text query in hybrid (text and vector) search. Previously, the maximum was fixed at 1,000. If you have more matches, you can now specify a higher limit to get more results back. <br><br>Third, for hybrid queries, you can set a weight on vector queries to have more or less importance than the nonvector query. |
+| [Vector results customization](vector-search-how-to-query.md) and [hybrid query optimizations](hybrid-search-how-to-query.md) | Feature | Three enhancements improve vector search relevance. <br><br>First, you can now set thresholds on vector search results to exclude low-scoring results. <br><br>Second, you can set `MaxTextRecallSize` and `countAndFacetMode` in hybrid queries to specify the maximum number of documents that can be recalled using text query in hybrid (text and vector) search. Previously, the maximum was fixed at `1000`, and the default is still `1000`. However, if you have more matches, you can now specify a higher limit on top and skip to get more results back. <br><br>Third, for hybrid queries queries, you can set a weight on vector queries to have more or less importance than the nonvector query. |
 | [Binary vectors support](/rest/api/searchservice/supported-data-types) | Feature | `Collection(Edm.Byte)` is a new supported data type. This data type opens up integration with the [Cohere v3 binary embedding models](https://cohere.com/blog/int8-binary-embeddings) and custom binary quantization. Narrow data types lower the cost of large vector datasets. See [Index binary data for vector search](vector-search-how-to-index-binary-data.md) for more information.| 
 | [Azure AI Vision multimodal embeddings skill (preview)](cognitive-search-skill-vision-vectorize.md) | Skill | New skill that's bound to the [multimodal embeddings API of Azure AI Vision](../ai-services/computer-vision/concept-image-retrieval.md). You can generate embeddings for text or images during indexing. This skill is available through the Azure portal and the [2024-05-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2024-05-01-preview&preserve-view=true).|
 | [Azure AI Vision vectorizer (preview)](vector-search-vectorizer-ai-services-vision.md) | Vectorizer | New vectorizer connects to an Azure AI Vision resource using the [multimodal embeddings API](../ai-services/computer-vision/concept-image-retrieval.md) to generate embeddings at query time. This vectorizer is available through the Azure portal and the [2024-05-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2024-05-01-preview&preserve-view=true). |