Skip to content

Commit 359feac

Browse files
Merge pull request #6001 from haileytap/freshness
{Azure Search] July freshness
2 parents f9b41ff + 64dc05e commit 359feac

File tree

2 files changed

+62
-47
lines changed

2 files changed

+62
-47
lines changed

articles/search/vector-search-filters.md

Lines changed: 55 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,50 +1,49 @@
11
---
2-
title: Vector query filters
2+
title: Vector Query Filters
33
titleSuffix: Azure AI Search
4-
description: Explains prefilters and post-filters in vector queries, and how filters affect query performance.
5-
4+
description: Learn how to add filter expressions to vector queries in Azure AI Search. Configure prefiltering and postfiltering modes to optimize query performance and improve search results.
65
author: haileytap
76
ms.author: haileytapia
87
ms.service: azure-ai-search
98
ms.update-cycle: 180-days
109
ms.custom:
11-
- ignite-2023
10+
+ ignite-2023
1211
ms.topic: how-to
13-
ms.date: 02/24/2025
12+
ms.date: 07/11/2025
1413
---
1514

16-
# Add a filter in a vector query in Azure AI Search
15+
# Add a filter to a vector query in Azure AI Search
1716

18-
You can define a vector query request that includes a [filter expression](search-filters.md) to add inclusion or exclusion criteria to your queries. In this article, learn how to:
17+
In Azure AI Search, you can define a vector query request that includes a [filter expression](search-filters.md) to add inclusion or exclusion criteria to your queries. This article explains how to:
1918

2019
> [!div class="checklist"]
21-
> - [Define a `filter` expression](#define-a-filter)
22-
> - [Set the `vectorFilterMode` for pre-query or post-query filtering](#set-the-vectorfiltermode)
20+
> + [Define a `filter` expression](#define-a-filter)
21+
> + [Set the `vectorFilterMode` for pre-query or post-query filtering](#set-the-vectorfiltermode)
2322
24-
This article uses REST for illustration. For code samples in other languages, see the [azure-search-vector-samples](https://github.com/Azure/azure-search-vector-samples) GitHub repository for end-to-end solutions that include vector queries.
23+
This article uses REST for illustration. For code samples in other languages and end-to-end solutions that include vector queries, see the [azure-search-vector-samples](https://github.com/Azure/azure-search-vector-samples) GitHub repository.
2524

2625
You can also use [Search Explorer](search-get-started-portal-import-vectors.md#check-results) in the Azure portal to query vector content. If you use the JSON view, you can add filters and specify the filter mode.
2726

2827
## How filtering works in a vector query
2928

30-
Filters apply to `filterable` *nonvector* fields, either a string field or numeric, to include or exclude search documents based on filter criteria. Although a vector field isn't filterable itself, filters can be applied to other nonvector fields in the same index, including or excluding the documents that also happen to contain vector fields you're searching on.
29+
Filters apply to `filterable` *nonvector* fields, either string or numeric, to include or exclude search documents based on filter criteria. Although a vector field isn't filterable itself, you can add filters to nonvector fields in the same index to include or exclude documents that also contain vector fields you're searching on.
3130

3231
Filters are applied before or after query execution based on the `vectorFilterMode` parameter.
3332

3433
## Define a filter
3534

36-
Filters determine the scope of a vector query. Filters are set on and iterate over nonvector string and numeric fields attributed as `filterable` in the index, but the purpose of a filter determines *what* the vector query executes over: the entire searchable space, or the contents of a search result.
35+
Filters determine the scope of a vector query. They're set on and iterate over nonvector string and numeric fields marked as `filterable` in the index. The purpose of a filter determines *what* the vector query executes over: the entire searchable space (prefiltering) or the contents of a search result (postfiltering).
3736

38-
If you don't have source fields with text or numeric values, check for document metadata, such as LastModified or CreatedBy properties, that might be useful in a metadata filter.
37+
If you don't have source fields with text or numeric values, check for document metadata, such as `LastModified` or `CreatedBy` properties, that might be useful in a metadata filter.
3938

4039
### [**2024-07-01**](#tab/filter-2024-07-01)
4140

42-
[**2024-07-01**](/rest/api/searchservice/search-service-api-versions#2024-07-01) is the stable version for this API. It has:
41+
[**2024-07-01**](/rest/api/searchservice/search-service-api-versions#2024-07-01) is the stable version of this API. This version has:
4342

44-
- `vectorFilterMode` for prefilter (default) or postfilter [filtering modes](vector-search-filters.md).
45-
- `filter` provides the criteria.
43+
+ `vectorFilterMode` for prefilter (default) or postfilter filtering modes.
44+
+ `filter` for the criteria.
4645

47-
In the following example, the vector is a representation of this query string: "what Azure services support full text search". The query targets the `contentVector` field. The actual vector has 1536 embeddings, so it's trimmed in this example for readability.
46+
In the following example, the vector is a representation of this query string: "what Azure services support full text search". The query targets the `contentVector` field. The actual vector has 1,536 embeddings, so it's trimmed for readability.
4847

4948
The filter criteria are applied to a filterable text field (`category` in this example) before the search engine executes the vector query.
5049

@@ -79,10 +78,10 @@ api-key: {{admin-api-key}}
7978

8079
[**2024-05-01-preview**](/rest/api/searchservice/search-service-api-versions#2024-05-01-preview) introduces filter options. This version adds:
8180

82-
- `vectorFilterMode` for prefilter (default) or postfilter [filtering modes](vector-search-filters.md).
83-
- `filter` provides the criteria.
81+
+ `vectorFilterMode` for prefilter (default) or postfilter filtering modes.
82+
+ `filter` for the criteria.
8483

85-
In the following example, the vector is a representation of this query string: "what Azure services support full text search". The query targets the `contentVector` field. The actual vector has 1536 embeddings, so it's trimmed in this example for readability.
84+
In the following example, the vector is a representation of this query string: "what Azure services support full text search". The query targets the `contentVector` field. The actual vector has 1,536 embeddings, so it's trimmed for readability.
8685

8786
The filter criteria are applied to a filterable text field (`category` in this example) before the search engine executes the vector query.
8887

@@ -117,33 +116,37 @@ api-key: {{admin-api-key}}
117116

118117
## Set the vectorFilterMode
119118

120-
The vectorFilterMode query parameter determines whether the filter is applied before or after vector query execution.
119+
The `vectorFilterMode` query parameter determines whether the filter is applied before or after vector query execution.
121120

122-
### Use prefilter mode
121+
### [Prefilter mode](#tab/prefilter-mode)
123122

124-
Prefiltering applies filters before query execution, reducing the search surface area over which the vector search algorithm looks for similar content.
123+
Prefiltering applies filters before query execution, reducing the search surface area over which the vector search algorithm looks for similar content.
125124

126125
In a vector query, `preFilter` is the default.
127126

128127
:::image type="content" source="media/vector-search-filters/pre-filter.svg" alt-text="Diagram of prefilters." border="true" lightbox="media/vector-search-filters/pre-filter.png":::
129128

130-
### Use postfilter mode
129+
### [Postfilter mode](#tab/postfilter-mode)
130+
131+
Postfiltering applies filters after query execution, narrowing the search results.
131132

132-
Post-filtering applies filters after query execution, narrowing the search results.
133+
In a vector query, use `postFilter` for this task.
133134

134135
:::image type="content" source="media/vector-search-filters/post-filter.svg" alt-text="Diagram of post-filters." border="true" lightbox="media/vector-search-filters/post-filter.png":::
135136

137+
---
138+
136139
### Benchmark testing of vector filter modes
137140

138141
To understand the conditions under which one filter mode performs better than the other, we ran a series of tests to evaluate query outcomes over small, medium, and large indexes.
139142

140-
+ Small (100,000 documents, 2.5-GB index, 1536 dimensions)
141-
+ Medium (1 million documents, 25-GB index, 1536 dimensions)
143+
+ Small (100,000 documents, 2.5-GB index, 1,536 dimensions)
144+
+ Medium (1 million documents, 25-GB index, 1,536 dimensions)
142145
+ Large (1 billion documents, 1.9-TB index, 96 dimensions)
143146

144147
For the small and medium workloads, we used a Standard 2 (S2) service with one partition and one replica. For the large workload, we used a Standard 3 (S3) service with 12 partitions and one replica.
145148

146-
Indexes had an identical construction: one key field, one vector field, one text field, and one numeric filterable field. The following index is defined using the 2023-11-03 syntax.
149+
Indexes had an identical construction: one key field, one vector field, one text field, and one numeric filterable field. The following index is defined using the `2023-11-03` syntax.
147150

148151
```python
149152
def get_index_schema(self, index_name, dimensions):
@@ -177,42 +180,54 @@ def get_index_schema(self, index_name, dimensions):
177180
}
178181
```
179182

180-
In queries, we used an identical filter for both prefilter and postfilter operations. We used a simple filter to ensure that variations in performance were due to filtering mode, and not filter complexity.
183+
In queries, we used an identical filter for both prefilter and postfilter operations. We used a simple filter to ensure that variations in performance were due to filtering mode, not filter complexity.
181184

182-
Outcomes were measured in Queries Per Second (QPS).
185+
Outcomes were measured in queries per second (QPS).
183186

184187
### Takeaways
185188

186189
+ Prefiltering is almost always slower than postfiltering, except on small indexes where performance is approximately equal.
187190

188191
+ On larger datasets, prefiltering is orders of magnitude slower.
189192

190-
+ So why is prefilter the default if it's almost always slower? Prefiltering guarantees that `k` results are returned if they exist in the index, where the bias favors recall and precision over speed.
193+
+ Why is prefilter the default if it's almost always slower? Prefiltering guarantees that `k` results are returned if they exist in the index, where the bias favors recall and precision over speed.
191194

192-
+ Postfiltering is for customers who:
195+
+ Use postfiltering if you:
193196

194-
+ value speed over selection (postfiltering can return fewer than `k` results)
195-
+ use filters that aren't overly selective
196-
+ have indexes of sufficient size such that prefiltering performance is unacceptable
197+
+ Value speed over selection (postfiltering can return fewer than `k` results).
198+
199+
+ Use filters that aren't overly selective.
200+
201+
+ Have indexes of sufficient size such that prefiltering performance is unacceptable.
197202

198203
### Details
199204

200-
+ Given a dataset with 100,000 vectors at 1536 dimensions:
205+
+ Given a dataset with 100,000 vectors at 1,536 dimensions:
206+
201207
+ When filtering more than 30% of the dataset, prefiltering and postfiltering were comparable.
208+
202209
+ When filtering less than 0.1% of the dataset, prefiltering was about 50% slower than postfiltering.
203210

204-
+ Given a dataset with 1 million vectors at 1536 dimensions:
211+
+ Given a dataset with 1 million vectors at 1,536 dimensions:
212+
205213
+ When filtering more than 30% of the dataset, prefiltering was about 30% slower.
214+
206215
+ When filtering less than 2% of the dataset, prefiltering was about seven times slower.
207216

208217
+ Given a dataset with 1 billion vectors at 96 dimensions:
218+
209219
+ When filtering more than 5% of the dataset, prefiltering was about 50% slower.
220+
210221
+ When filtering less than 10% of the dataset, prefiltering was about seven times slower.
211222

212-
The following graph shows prefilter relative QPS, computed as prefilter QPS divided by postfilter QPS.
223+
The following graph shows prefilter relative QPS, computed as prefilter QPS divided by postfilter QPS.
213224

214225
:::image type="content" source="media/vector-search-filters/chart.svg" alt-text="Chart showing QPS performance for small, medium, and large indexes for relative QPS." border="true" lightbox="media/vector-search-filters/chart.png":::
215226

216-
The vertical axis is QPS of prefiltering over QPS of postfiltering. For example, a value of 0.0 means prefiltering is 100% slower, 0.5 on the vertical axis means prefiltering is 50% slower, 1.0 means prefiltering and post filtering are equivalent.
227+
The vertical axis represents the relative performance of prefiltering compared to postfiltering, expressed as a ratio of QPS (queries per second). For example:
228+
229+
+ A value of `0.0` means prefiltering is 100% slower than postfiltering.
230+
+ A value of `0.5` means prefiltering is 50% slower.
231+
+ A value of `1.0` means prefiltering and post filtering are equivalent.
217232

218-
The horizontal axis represents the filtering rate, or the percentage of candidate documents after applying the filter. For example, `1.00%` means that one percent of the search corpus was selected by the filter criteria.
233+
The horizontal axis represents the filtering rate, or the percentage of candidate documents after applying the filter. For example, a rate of `1.00%` means the filter criteria selected one percent of the search corpus.

articles/search/vector-search-how-to-query.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Vector query how-to
2+
title: Create a Vector Query
33
titleSuffix: Azure AI Search
44
description: Learn how to build queries for vector search.
55

@@ -15,14 +15,14 @@ ms.date: 06/19/2025
1515

1616
# Create a vector query in Azure AI Search
1717

18-
In Azure AI Search, if you have a [vector index](vector-search-how-to-create-index.md), this article explains how to:
18+
If you have a [vector index](vector-search-how-to-create-index.md) in Azure AI Search, this article explains how to:
1919

2020
> [!div class="checklist"]
21-
> + [Query vector fields](#vector-query-request).
22-
> + [Query multiple vector fields at once](#multiple-vector-fields).
23-
> + [Set vector weights](#vector-weighting).
24-
> + [Query with integrated vectorization](#query-with-integrated-vectorization).
25-
> + [Set thresholds to exclude low-scoring results (preview)](#set-thresholds-to-exclude-low-scoring-results-preview).
21+
> + [Query vector fields](#vector-query-request)
22+
> + [Query multiple vector fields at once](#multiple-vector-fields)
23+
> + [Set vector weights](#vector-weighting)
24+
> + [Query with integrated vectorization](#query-with-integrated-vectorization)
25+
> + [Set thresholds to exclude low-scoring results (preview)](#set-thresholds-to-exclude-low-scoring-results-preview)
2626
2727
This article uses REST for illustration. After you understand the basic workflow, continue with the Azure SDK code samples in the [azure-search-vector-samples](https://github.com/Azure/azure-search-vector-samples) repo, which provides end-to-end solutions that include vector queries.
2828

0 commit comments

Comments
 (0)