Skip to content

Commit 0ac1085

Browse files
committed
checkpoint
1 parent 365f4e7 commit 0ac1085

File tree

7 files changed

+119
-28
lines changed

7 files changed

+119
-28
lines changed
18.2 KB
Loading
101 KB
Loading
33.5 KB
Loading

articles/search/vector-search-filters.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,14 @@ ms.service: cognitive-search
99
ms.custom:
1010
- ignite-2023
1111
ms.topic: conceptual
12-
ms.date: 11/01/2023
12+
ms.date: 02/14/2024
1313
---
1414

1515
# Filters in vector queries
1616

17-
You can set a [vector filter modes on a vector query](vector-search-how-to-query.md) to specify whether you want filtering before or after query execution. Filters are set on and iterate over string and numeric fields attributed as `filterable` in the index, but the effects of filtering determine *what* the vector query executes over: the searchable space, or the documents in the search results.
17+
You can set a [**vector filter modes on a vector query**](vector-search-how-to-query.md) to specify whether you want filtering before or after query execution.
18+
19+
Filters set the scope of a vector query. Filter are set on and iterate over nonvector string and numeric fields attributed as `filterable` in the index, but the effects of filtering determine *what* the vector query executes over: the searchable space, or the contents of the search results.
1820

1921
This article describes each filter mode and provides guidance on when to use each one.
2022

@@ -40,21 +42,21 @@ To understand the conditions under which one filter mode performs better than th
4042

4143
For the small and medium workloads, we used a Standard 2 (S2) service with one partition and one replica. For the large workload, we used a Standard 3 (S3) service with 12 partitions and one replica.
4244

43-
Indexes had an identical construction: one key field, one vector field, one text field, and one numeric filterable field.
45+
Indexes had an identical construction: one key field, one vector field, one text field, and one numeric filterable field. The following index is defined using the 2023-07-01-preview syntax.
4446

4547
```python
4648
def get_index_schema(self, index_name, dimensions):
4749
return {
4850
"name": index_name,
4951
"fields": [
5052
{"name": "id", "type": "Edm.String", "key": True, "searchable": True},
51-
{"name": "myvector", "type": "Collection(Edm.Single)", "dimensions": dimensions,
53+
{"name": "content_vector", "type": "Collection(Edm.Single)", "dimensions": dimensions,
5254
"searchable": True, "retrievable": True, "filterable": False, "facetable": False, "sortable": False,
5355
"vectorSearchConfiguration": "defaulthnsw"},
5456
{"name": "text", "type": "Edm.String", "searchable": True, "filterable": False, "retrievable": True,
55-
"sortable": False, "facetable": False, "key": False},
57+
"sortable": False, "facetable": False},
5658
{"name": "score", "type": "Edm.Double", "searchable": False, "filterable": True,
57-
"retrievable": True, "sortable": True, "facetable": True, "key": False}
59+
"retrievable": True, "sortable": True, "facetable": True}
5860
],
5961
"vectorSearch":
6062
{

articles/search/vector-search-how-to-create-index.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -696,6 +696,22 @@ api-key: {{admin-api-key}}
696696

697697
---
698698

699+
## Update a vector store
700+
701+
To update a vector store, modify the schema and if necessary, reload documents to populate new fields. APIs for schema updates include [Create or Update Index (REST)](/rest/api/searchservice/indexes/create-or-update), [CreateOrUpdateIndex](/dotnet/api/azure.search.documents.indexes.searchindexclient.createorupdateindexasync) in the Azure SDK for .NET, [create_or_update_index](/python/api/azure-search-documents/azure.search.documents.indexes.searchindexclient?view=azure-python#azure-search-documents-indexes-searchindexclient-create-or-update-index&preserve-view=true) in the Azure SDK for Python, and similar methods in other Azure SDKs.
702+
703+
The standard guidance for updating an index is covered in [Drop and rebuild an index](search-howto-reindex.md).
704+
705+
Key points include:
706+
707+
+ Drop and rebuild is often required for updates to and deletion of existing fields.
708+
709+
+ However, you can update an existing schema with the following modifications, with no rebuild required:
710+
711+
+ Add new fields to a fields collection.
712+
+ Add new vector configurations, assigned to new fields but not existing fields that have already been vectorized.
713+
+ Change "retrievable" (values are true or false) on an existing field. Vector fields must be searchable and retrievable, but if you want to disable access to a vector field in situations where drop and rebuild isn't feasible, you can set retrievable to false.
714+
699715
## Next steps
700716

701717
As a next step, we recommend [Query vector data in a search index](vector-search-how-to-query.md).

articles/search/vector-search-index-size.md

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.service: cognitive-search
99
ms.custom:
1010
- ignite-2023
1111
ms.topic: conceptual
12-
ms.date: 01/30/2024
12+
ms.date: 02/14/2024
1313
---
1414

1515
# Vector index size limits
@@ -41,12 +41,48 @@ The following table shows vector quotas by partition, and by service if all part
4141

4242
+ Vector quotas for are the vector indexes created for each vector field, and they're enforced at the partition level. On Basic, the sum total of all vector fields can't be more than 1 GB because Basic only has one partition. On S1, which can have up to 12 partitions, the quota for vector data is 3 GB if you allocate just one partition, or up to 36 GB if you allocate all 12 partitions. For more information about partitions and replicas, see [Estimate and manage capacity](search-capacity-planning.md).
4343

44+
## How to determine service creation date
45+
46+
Find out whether your search service was created before July 1, 2023. If it's an older service, consider creating a new search service to benefit from the higher limits. Newer services at the same tier offer at least twice as much vector storage.
47+
48+
1. In Azure portal, open the resource group.
49+
50+
1. On the left nav pane, under **Settings**, select **Deployments**.
51+
52+
1. Locate your search service deployment. If there are many deployments, use the filter to look for "search".
53+
54+
1. Select the deployment. If you have more than one, click through to see if it resolves to your search service.
55+
56+
:::image type="content" source="media/vector-search-index-size/resource-group-deployments.png" alt-text="Screenshot of a filtered deployments list.":::
57+
58+
1. Expand deployment details. You should see *Created* and the creation date.
59+
60+
:::image type="content" source="media/vector-search-index-size/deployment-details.png" alt-text="Screenshot of the deployment details showing creation date.":::
61+
62+
1. Now that you know the age of your search service, review the vector quota limits based on service creation:
63+
64+
+ [Before July 1, 2023](search-limits-quotas-capacity.md#services-created-before-july-1-2023)
65+
+ [After July 1, 2023](search-limits-quotas-capacity.md#services-created-after-july-1-2023-in-supported-regions)
66+
4467
## How to get vector index size
4568

46-
Use the REST APIs to return vector index size:
69+
A request for vector metrics is a data plane operation. You can use the Azure portal, REST APIs, or Azure SDKs to get vector usage at the service level through service statistics and for individual indexes.
4770

48-
+ [GET Index Statistics](/rest/api/searchservice/indexes/get-statistics) returns usage for a given index.
71+
### [**Portal**](#tab/portal-vector-quota)
72+
73+
Usage information can be found on the **Overview** page's **Usage** tab. Portal pages refresh every few minutes so if you recently updated an index, wait a bit before checking results.
74+
75+
The following screenshot is for a newer Standard 1 (S1) tier, configured for one partition and one replica. Vector index quota, measured in megabytes, refers to the internal vector indexes created for each vector field. Overall, indexes consume almost 460 megabytes of available storage, but the vector index component takes up just 93 megabytes of the 460 used on this search service.
76+
77+
:::image type="content" source="media/vector-search-index-size/portal-vector-index-size.png" alt-text="Screenshot of the Overview page's usage tab showing vector index consumption against quota.":::
78+
79+
The tile on the Usage tab tracks vector index consumption at the search service level. If you increase or decrease search service capacity, the tile reflects the changes accordingly.
4980

81+
### [**REST**](#tab/rest-vector-quota)
82+
83+
Use the following data plane REST APIs (version 2023-11-01 or later) for vector usage statistics:
84+
85+
+ [GET Index Statistics](/rest/api/searchservice/indexes/get-statistics) returns usage for a given index.
5086
+ [GET Service Statistics](/rest/api/searchservice/get-service-statistics/get-service-statistics) returns quota and usage for the search service all-up.
5187

5288
For a visual, here's the sample response for a Basic search service that has the quickstart vector search index. `storageSize` and `vectorIndexSize` are reported in bytes.
@@ -94,6 +130,8 @@ Return service statistics to compare usage against available quota at the servic
94130
}
95131
```
96132

133+
---
134+
97135
## Factors affecting vector index size
98136

99137
There are three major components that affect the size of your internal vector index:

0 commit comments

Comments
 (0)