Merge pull request #3802 from HeidiSteen/release-azs-4

[release-azure-search] ENG feedback on facets and vector storage

Author: prmerger-automator[bot]

articles/search/search-faceted-navigation-examples.md

@@ -9,13 +9,15 @@ author: HeidiSteen
 ms.author: heidist
 ms.service: azure-ai-search
 ms.topic: how-to
-ms.date: 03/21/2025
+ms.date: 03/31/2025
 ---
 
 # Faceted navigation examples
 
 This section extends [faceted navigation configuration](search-faceted-navigation.md) with examples that demonstrate basic usage and other scenarios.
 
+Facetable fields are defined in an index, but facet parameters and expressions are defined in query requests. If you have an index with facetable fields, you can try new features like [facet hierarchs](#facet-hierarchy-example) and [aggregations](#facet-aggregation-example) on existing indexes.
+
 ## Facet parameters and syntax
 
 Depending on the API, a facet query is usually an array of facet expressions that are applied to search results. Each facet expression contains a facetable field name, optionally followed by a comma-separated list of name-value pairs.
@@ -202,22 +204,22 @@ Results from this query are as follows:
 
 Starting in [2025-03-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2025-03-01-preview&preserve-view=true) and available in the Azure portal, you can configure a facet hierarchy using the `>` and `;` operators.
 
-The nesting (hierarchical) operator `>` denotes a parent–child relationship, and the semicolon operator `;` denotes children of a shared parent. The parent must contain only one field. Both the parent and child fields must be facetable. 
+The nesting (hierarchical) operator `>` denotes a parent–child relationship, and the semicolon operator `;` denotes multiple fields at the same nesting level, which are all children of the same parent. The parent must contain only one field. Both the parent and child fields must be `facetable`. 
 
 The order of operations in a facet expression that includes facet hierarchies are:
 
 * options operator (comma `,`) that separates facet parameters for the facet field, such as the comma in `Rooms/BaseRate,values`
-* parentheses, such as the ones enclosing `Rooms/BaseRate`.
+* parentheses, such as the ones enclosing `(Rooms/BaseRate,values:50 ; Rooms/Type)`.
 * nesting operator (angled bracket `>`)
-* append operator (semicolon `;`), demonstrated in a second example `"Tags>(Rooms/BaseRate,values:50;Rooms/Type)"` in this section, where two child facets are peers under the Tags parent.
+* append operator (semicolon `;`), demonstrated in a second example `"Tags>(Rooms/BaseRate,values:50 ; Rooms/Type)"` in this section, where two child facets are peers under the Tags parent.
 
-Here's a query that returns just a few documents, which is helpful for viewing a full response. Facets count the parent document (Hotels) and not intermediate subdocuments (Rooms), so the response determines the number of *hotels* that have any rooms in each facet bucket.
+There are several examples for facet hierarchies. The first example is a query that returns just a few documents, which is helpful for viewing a full response. Facets count the parent document (Hotels) and not intermediate subdocuments (Rooms), so the response determines the number of *hotels* that have any rooms in each facet bucket.
 
 ```rest
 POST /indexes/hotels-sample-index/docs/search?api-version=2025-03-01-Preview
 {
-  "search": "+ocean",  
-  "facets": ["Address/StateProvince>Address/City", "Tags>(Rooms/BaseRate,values:50)"],
+  "search": "ocean",  
+  "facets": ["Address/StateProvince>Address/City", "Tags>Rooms/BaseRate,values:50"],
   "select": "HotelName, Description, Tags, Address/StateProvince, Address/City",
   "count": true 
 }
@@ -371,13 +373,13 @@ Results from this query are as follows. Both hotels have pools. For other tags,
 }
 ```
 
-This example extends the previous one, demonstrating multiple top-level facets with multiple children. Notice the semicolon (`;`) operator separates each child.
+This second example extends the previous one, demonstrating multiple top-level facets with multiple children. Notice the semicolon (`;`) operator separates each child.
 
 ```rest
 POST /indexes/hotels-sample-index/docs/search?api-version=2025-03-01-Preview
 {  
   "search": "+ocean",  
-  "facets": ["Address/StateProvince>Address/City", "Tags>(Rooms/BaseRate,values:50;Rooms/Type)"],
+  "facets": ["Address/StateProvince > Address/City", "Tags > (Rooms/BaseRate,values:50 ; Rooms/Type)"],
   "select": "HotelName, Description, Tags, Address/StateProvince, Address/City",
   "count": true 
 }  
@@ -427,6 +429,50 @@ A partial response, trimmed for brevity, shows Tags with child facets for the ro
 }
 ```
 
+This last example shows precedence rules for parentheses that affects nesting levels. Suppose you want to return a facet hierarchy in this order.
+
+```
+Address/StateProvince
+  Address/City
+    Category
+    Rating
+```
+
+To return this hierarchy, create a query where Category and Rating are siblings under Address/City.
+
+```json
+  { 
+    "search": "beach",  
+    "facets": [
+        "Address/StateProvince > (Address/City > (Category ; Rating))"
+        ],
+    "select": "HotelName, Description, Tags, Address/StateProvince, Address/City",
+    "count": true 
+  }
+```
+
+If you remove the innermost parentheses, Category and Rating are no longer siblings because the precedence rules mean that the `>` operator is evaluated before `;`.
+
+```json
+  { 
+    "search": "beach",  
+    "facets": [
+        "Address/StateProvince > (Address/City > Category ; Rating)"
+        ],
+    "select": "HotelName, Description, Tags, Address/StateProvince, Address/City",
+    "count": true 
+  }
+```
+
+The top-level parent is still Address/StateProvince, but now Address/City and Rating are on same level.
+
+```
+Address/StateProvince
+  Rating
+  Address/City
+    Category
+```
+
 ## Facet filtering example
 
 [!INCLUDE [Feature preview](./includes/previews/preview-generic.md)]
@@ -438,7 +484,7 @@ Facet filtering enables you to constrain the facet values returned to those matc
 * `includeTermFilter` filters the facet values to those that match the regular expression
 * `excludeTermFilter` filters the facet values to those that don't match the regular expression 
 
-If a facet string satisfies both conditions, the `excludeTermFilter` takes precedence. Otherwise, the set of bucket strings are first evaluated with `includeTermFilter` and then excluded with `excludeTermFilter`.
+If a facet string satisfies both conditions, the `excludeTermFilter` takes precedence because the set of bucket strings is first evaluated with `includeTermFilter` and then excluded with `excludeTermFilter`.
 
 Only those facet values that match the regular expression are returned. You can combine these parameters with other facet options (for example, `count`, `sort`, and [hierarchical faceting](#facet-hierarchy-example)) on string fields.
 
@@ -449,7 +495,7 @@ The following example shows how to escape special characters in your regular exp
 ```json
 {
     "search": "*", 
-    "facets": ["name,includeTermFilter:/EscapeBackslash\\OrDoubleQuote\\"OrRegexCharacter\\(/"] 
+    "facets": ["name,includeTermFilter:/EscapeBackslash\\\OrDoubleQuote\\"OrRegexCharacter\\(/"] 
 }
 ```
 
@@ -556,7 +602,9 @@ The following example is an abbreviated response (hotel documents are omitted fo
 
 Starting in [2025-03-01-preview REST API](/rest/api/searchservice/operation-groups?view=rest-searchservice-2025-03-01-preview&preserve-view=true) and available in the Azure portal, you can aggregate facets.
 
-Facet aggregations allow you to compute metrics from facet values. The aggregation capability works alongside the existing faceting options. The only supported metric is `sum`. Adding `metric: sum` to a numeric facet aggregates all the values of each bucket.
+Facet aggregations allow you to compute metrics from facet values. The aggregation capability works alongside the existing faceting options. The only supported metric is `sum`. Adding `metric: sum` to a numeric facet aggregates all the values of each bucket. 
+
+You can add a default value to use if a document contains a null for that field: `"facets": [ "Rooms/SleepsCount, metric: sum, default:2"]`. If a room has a null value for the Rooms/SleepsCount field, the default substitutes for the missing value.
 
 You can sum any facetable field of a numeric data type (except vectors and geographic coordinates). 
 

articles/search/search-faceted-navigation.md

@@ -8,7 +8,7 @@ author: HeidiSteen
 ms.author: heidist
 ms.service: azure-ai-search
 ms.topic: how-to
-ms.date: 03/21/2025
+ms.date: 03/31/2025
 ---
 
 # Add faceted navigation to search results
@@ -121,7 +121,7 @@ Facets can be calculated over single-value fields and collections. Fields that w
 * Low cardinality (a few distinct values that repeat throughout documents in your search corpus).
 * Short descriptive values (one or two words) that render nicely in a navigation tree.
 
-The values within a field, and not the field name itself, produce the facets in a faceted navigation structure. If the facet is a string field named *Color*, facets are blue, green, and any other value for that field. As a best practice, review field values to ensure there are no typos, nulls, or casing differences. Consider [assigning a normalizer](search-normalizers.md) to a "filterable" and "facetable" field to smooth out minor variations in the text.
+The values within a field, and not the field name itself, produce the facets in a faceted navigation structure. If the facet is a string field named *Color*, facets are blue, green, and any other value for that field. As a best practice, review field values to ensure there are no typos, nulls, or casing differences. Consider [assigning a normalizer](search-normalizers.md) to a filterable and facetable field to smooth out minor variations in the text. For example, "Canada", "CANADA", and "canada" would all be normalized to one bucket.
 
 You can't set facets on existing fields, on vector fields, or fields of type `Edm.GeographyPoint` or `Collection(Edm.GeographyPoint)`.
 
@@ -220,7 +220,7 @@ Here's a screenshot of the [basic facet query example](search-faceted-navigation
     | `interval` | An integer interval greater than zero for numbers, or minute, hour, day, week, month, quarter, year for date time values. For example, `"facet=baseRate,interval:100"` produces buckets based on base rate ranges of size 100. If base rates are all between $60 and $600, there are buckets for 0-100, 100-200, 200-300, 300-400, 400-500, and 500-600. The string `"facet=lastRenovationDate,interval:year"` produces one bucket for each year when hotels were renovated. |
     | `timeoffset` | Can be set to (`[+-]hh:mm, [+-]hhmm, or [+-]hh`). If used, the `timeoffset` parameter must be combined with the interval option, and only when applied to a field of type `Edm.DateTimeOffset`. The value specifies the UTC time offset to account for in setting time boundaries. For example: `"facet=lastRenovationDate,interval:day,timeoffset:-01:00"` uses the day boundary that starts at 01:00:00 UTC (midnight in the target time zone). |
 
-`count` and `sort` can be combined in the same facet specification, but they can't be combined with interval or values, and interval and values can't be combined together.
+`count` and `sort` can be combined in the same facet specification, but they can't be combined with `interval` or `values`, and `interval` and `values` can't be combined together.
 
 Interval facets on date time are computed based on the UTC time if `timeoffset` isn't specified. For example, for `"facet=lastRenovationDate,interval:day"`, the day boundary starts at 00:00:00 UTC.
 
@@ -250,6 +250,10 @@ Remember that you can't use `Edm.GeographyPoint` or `Collection(Edm.GeographyPoi
 
 As you prepare data for indexing, check fields for null values, misspellings or case discrepancies, and single and plural versions of the same word. By default, filters and facets don't undergo lexical analysis or [spell check](speller-how-to-add.md), which means that all values of a "facetable" field are potential facets, even if the words differ by one character. Optionally, you can [assign a normalizer](search-normalizers.md) to a "filterable" and "facetable" field to smooth out variations in casing and characters.
 
+### Ordering facet buckets
+
+Although you can sort within a bucket, there's no parameters for controlling the order of facet buckets in the navigation structure as a whole. If you want facet buckets in a specific order, you must provide it in application code.
+
 ### Discrepancies in facet counts
 
 Under certain circumstances, you might find that facet counts aren't fully accurate due to the [sharding architecture](index-similarity-and-scoring.md#sharding-effects-on-query-results). Every search index is spread across multiple shards, and each shard reports the top N facets by document count, which are then combined into a single result. Because it's just the top N facets for each shard, it's possible to miss or under-count matching documents in the facet response.

articles/search/vector-search-how-to-chunk-documents.md

@@ -9,7 +9,7 @@ ms.service: azure-ai-search
 ms.custom:
   - ignite-2023
 ms.topic: conceptual
-ms.date: 03/11/2025
+ms.date: 03/31/2025
 ---
 
 # Chunk large documents for vector search solutions in Azure AI Search
@@ -20,7 +20,9 @@ We recommend [integrated vectorization](vector-search-integrated-vectorization.m
 
 ## Common chunking techniques
 
-Chunking is only required if the source documents are too large for the maximum input size imposed by models. Here are some common chunking techniques, associated with built-in features if you use [indexers](search-indexer-overview.md) and [skills](cognitive-search-working-with-skillsets.md).
+Chunking is only required if the source documents are too large for the maximum input size imposed by models, but it's also beneficial if content is poorly represented as a single vector. Consider a wiki page that covers a lot of varied sub-topics. The entire page might be small enough to meet model input requirements, but you might get better results if you chunk at a finer grain.
+
+Here are some common chunking techniques, associated with built-in features if you use [indexers](search-indexer-overview.md) and [skills](cognitive-search-working-with-skillsets.md).
 
 | Approach | Usage | Built-in functionality |
 |----------|-------|-----------------|

articles/search/vector-search-how-to-quantization.md

@@ -52,15 +52,15 @@ Rescoring is technique used to offset information loss due to vector compression
 Rescoring applies to:
 
 - scalar quantization using Hierarchical Navigable Small World (HNSW) graphs for similarity search
-- binary quantization using HNSW graphs
+- binary quantization, also using HNSW graphs
 
 Exhaustive K Nearest Neighbors (eKNN) doesn't support rescoring.
 
 Rescoring occurs when you set a rescoring option in the index vector configuration:
 
 - In version 2024-07-01, set `rerankWithOriginalVectors`
 - In version 2024-11-01-preview, set `rescoringOptions.enableRescoring` and `rescoreStorageMethod.preserveOriginals`
-- In version 2025-03-01-preview, set `rescoringOptions.enableRescoring` and `rescoringOptions.rescoreStorageMethod=preserveOriginals` for scalar quantization, or `rescoringOptions.enableRescoring` for binary quantization.
+- In version 2025-03-01-preview, set `rescoringOptions.enableRescoring` and `rescoringOptions.rescoreStorageMethod=preserveOriginals` for scalar or binary quantization, or `rescoringOptions.enableRescoring` and `rescoringOptions.rescoreStorageMethod=discardOriginals` for binary quantization only
 
 The generalized process for rescoring is:
 
@@ -372,11 +372,11 @@ Each component of the vector is mapped to the closest representative value withi
 
 Binary quantization compresses high-dimensional vectors by representing each component as a single bit, either 0 or 1. This method drastically reduces the memory footprint and accelerates vector comparison operations, which are crucial for search and retrieval tasks. Benchmark tests show up to 96% reduction in vector index size.
 
-It's particularly effective for embeddings with dimensions greater than 1024. For smaller dimensions, we recommend testing the quality of binary quantization, or trying scalar instead. Additionally, we’ve found BQ performs very well when embeddings are centered around zero. Most popular embedding models such as OpenAI, Cohere, and Mistral are centered around zero.
+It's particularly effective for embeddings with dimensions greater than 1024. For smaller dimensions, we recommend testing the quality of binary quantization, or trying scalar instead. Additionally, we’ve found binary quantization performs very well when embeddings are centered around zero. Most popular embedding models such as OpenAI, Cohere, and Mistral are centered around zero.
 
 ## Query a quantized vector field using oversampling
 
-Query syntax for a compressed or quantized vector field is the same as for noncompressed vector fields, unless you want to override parameters associated with oversampling or rescoring with original vectors.
+Query syntax for a compressed or quantized vector field is the same as for noncompressed vector fields, unless you want to override parameters associated with oversampling and rescoring. You can add an o`versampling` parameter to invoke oversampling and rescoring at query time.
 
 ### [**2024-07-01**](#tab/query-2024-07-01)
 
@@ -430,9 +430,9 @@ POST https://[service-name].search.windows.net/indexes/demo-index/docs/search?ap
 
 **Key points**:
 
-- Applies to vector fields that undergo vector compression, per the vector profile assignment.
+- Oversampling applies to vector fields that undergo vector compression, per the vector profile assignment.
 
-- Overrides the `defaultOversampling` value or introduces oversampling at query time, even if the index's compression configuration didn't specify oversampling or reranking options.
+- Oversampling in the query overrides the `defaultOversampling` value in the index, or invokes oversampling and rescoring at query time, even if the index's compression configuration didn't specify oversampling or reranking options.
 
 ### [**2025-03-01-preview**](#tab/query-2025-03-01-preview)
 
@@ -454,4 +454,10 @@ POST https://[service-name].search.windows.net/indexes/demo-index/docs/search?ap
 }
 ```
 
+**Key points**:
+
+- Oversampling applies to vector fields that undergo vector compression, per the vector profile assignment.
+
+- Oversampling in the query overrides the `defaultOversampling` value in the index, or invokes oversampling and rescoring at query time, even if the index's compression configuration didn't specify oversampling or reranking options.
+
 ---