Skip to content

Commit 5f466bb

Browse files
garvit-guptaOxyjun
andauthored
[Vectorize] Add documentation for the Vectorize list-vectors operation (#24680)
--------- Co-authored-by: Jun Lee <[email protected]>
1 parent d103102 commit 5f466bb

File tree

5 files changed

+103
-1
lines changed

5 files changed

+103
-1
lines changed
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: List vectors
3+
pcx_content_type: concept
4+
sidebar:
5+
order: 5
6+
7+
---
8+
9+
The list-vectors operation allows you to enumerate all vector identifiers in a Vectorize index using paginated requests. This guide covers best practices for efficiently using this operation.
10+
11+
## When to use list-vectors
12+
13+
Use list-vectors for:
14+
15+
- **Bulk operations**: To process all vectors in an index
16+
- **Auditing**: To verify the contents of your index or generate reports
17+
- **Data migration**: To move vectors between indexes or systems
18+
- **Cleanup operations**: To identify and remove outdated vectors
19+
20+
## Pagination behavior
21+
22+
The list-vectors operation uses cursor-based pagination with important consistency guarantees:
23+
24+
### Snapshot consistency
25+
26+
Vector identifiers returned belong to the index snapshot captured at the time of the first list-vectors request. This ensures consistent pagination even when the index is being modified during iteration:
27+
28+
- **New vectors**: Vectors inserted after the initial request will not appear in subsequent paginated results
29+
- **Deleted vectors**: Vectors deleted after the initial request will continue to appear in the remaining responses until pagination is complete
30+
31+
### Starting a new iteration
32+
33+
To see recently added or removed vectors, you must start a new list-vectors request sequence (without a cursor). This captures a fresh snapshot of the index.
34+
35+
### Response structure
36+
37+
Each response includes:
38+
- `count`: Number of vectors returned in this response
39+
- `totalCount`: Total number of vectors in the index
40+
- `isTruncated`: Whether there are more vectors available
41+
- `nextCursor`: Cursor for the next page (null if no more results)
42+
- `cursorExpirationTimestamp`: Timestamp of when the cursor expires
43+
- `vectors`: Array of vector identifiers
44+
45+
### Cursor expiration
46+
47+
Cursors have an expiration timestamp. If a cursor expires, you'll need to start a new list-vectors request sequence to continue pagination.
48+
49+
## Performance considerations
50+
51+
Take care to have sufficient gap between consecutive requests to avoid hitting rate-limits.
52+
53+
## Example workflow
54+
55+
Here's a typical pattern for processing all vectors in an index:
56+
57+
```sh
58+
# Start iteration
59+
wrangler vectorize list-vectors my-index --count=1000
60+
61+
# Continue with cursor from response
62+
wrangler vectorize list-vectors my-index --count=1000 --cursor="<cursor-from-response>"
63+
64+
# Repeat until no more results
65+
```

src/content/docs/vectorize/platform/limits.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,13 @@ The following limits apply to accounts, indexes and vectors (as specified):
1111
| ------------------------------------------------------------- | ----------------------------------- |
1212
| Indexes per account | 50,000 (Workers Paid) / 100 (Free) |
1313
| Maximum dimensions per vector | 1536 dimensions, 32 bits precision |
14-
| Precision per vector dimension | 32 bits (float32) |
14+
| Precision per vector dimension | 32 bits (float32) |
1515
| Maximum vector ID length | 64 bytes |
1616
| Metadata per vector | 10KiB |
1717
| Maximum returned results (`topK`) with values or metadata | 20 |
1818
| Maximum returned results (`topK`) without values and metadata | 100 |
1919
| Maximum upsert batch size (per batch) | 1000 (Workers) / 5000 (HTTP API) |
20+
| Maximum vectors in a list-vectors page | 1000 |
2021
| Maximum index name length | 64 bytes |
2122
| Maximum vectors per index | 5,000,000 |
2223
| Maximum namespaces per index | 50,000 (Workers Paid) / 1000 (Free) |

src/content/docs/vectorize/reference/client-api.mdx

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,22 @@ const details = await env.YOUR_INDEX.describe();
127127

128128
Retrieves the configuration of a given index directly, including its configured `dimensions` and distance `metric`.
129129

130+
### List Vectors
131+
132+
List all vector identifiers in an index using paginated requests, returning up to 1000 vector identifiers per page.
133+
134+
```sh
135+
wrangler vectorize list-vectors <index-name> [--count=<number>] [--cursor=<cursor-string>]
136+
```
137+
138+
**Parameters:**
139+
140+
- `<index-name>` - The name of your Vectorize index
141+
- `--count` (optional) - Number of vector IDs to return per page. Must be between 1 and 1000 (default: 100)
142+
- `--cursor` (optional) - Pagination cursor from the previous response to continue listing from that position
143+
144+
For detailed guidance on pagination behavior and best practices, refer to [List vectors best practices](/vectorize/best-practices/list-vectors/).
145+
130146
### Create Metadata Index
131147

132148
Enable metadata filtering on the specified property. Limited to 10 properties.

src/content/partials/workers/wrangler-commands/vectorize.mdx

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,21 @@ npx wrangler vectorize query <INDEX_NAME> [OPTIONS]
130130
- `--filter` <Type text="string" /> <MetaInfo text="optional" />
131131
- Filter vectors based on this metadata filter. Example: `'{ 'p1': 'abc', 'p2': { '$ne': true }, 'p3': 10, 'p4': false, 'nested.p5': 'abcd' }'`
132132

133+
<AnchorHeading title="`list-vectors`" slug="vectorize-list-vectors" depth={3} />
134+
135+
List vector identifiers in a Vectorize index in a paginated manner.
136+
137+
```sh
138+
npx wrangler vectorize list-vectors <INDEX_NAME> [OPTIONS]
139+
```
140+
141+
- `INDEX_NAME` <Type text="string" /> <MetaInfo text="required" />
142+
- The name of the Vectorize index from which vector identifiers need to be listed.
143+
- `--count` <Type text="number" /> <MetaInfo text="optional" />
144+
- Number of vector IDs to return per page. Must be between 1 and 1000 (default: `100`).
145+
- `--cursor` <Type text="string" /> <MetaInfo text="optional" />
146+
- Pagination cursor from the previous response to continue listing from that position.
147+
133148
<AnchorHeading title="`get-vectors`" slug="vectorize-get-vectors" depth={3} />
134149

135150
Fetch vectors from a Vectorize index using the provided ids.

src/content/release-notes/vectorize.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@ link: "/vectorize/platform/changelog/"
33
productName: Vectorize
44
productLink: "/vectorize/"
55
entries:
6+
- publish_date: "2025-08-25"
7+
title: Added support for the list-vectors operation
8+
description: |-
9+
Vectorize now supports iteration through all the vector identifiers in an index in a paginated manner using the list-vectors operation.
10+
611
- publish_date: "2024-12-20"
712
title: Added support for index name reuse
813
description: |-

0 commit comments

Comments
 (0)