Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions src/content/docs/vectorize/best-practices/list-vectors.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: List vectors
pcx_content_type: concept
sidebar:
order: 5

---

The list-vectors operation allows you to enumerate all vector identifiers in a Vectorize index using paginated requests. This guide covers best practices for efficiently using this operation.

## When to use list-vectors

Use list-vectors for:

- **Bulk operations**: To process all vectors in an index
- **Auditing**: To verify the contents of your index or generate reports
- **Data migration**: To move vectors between indexes or systems
- **Cleanup operations**: To identify and remove outdated vectors

## Pagination behavior

The list-vectors operation uses cursor-based pagination with important consistency guarantees:

### Snapshot consistency

Vector identifiers returned belong to the index snapshot captured at the time of the first list-vectors request. This ensures consistent pagination even when the index is being modified during iteration:

- **New vectors**: Vectors inserted after the initial request will not appear in subsequent paginated results
- **Deleted vectors**: Vectors deleted after the initial request will continue to appear in the remaining responses until pagination is complete

### Starting a new iteration

To see recently added or removed vectors, you must start a new list-vectors request sequence (without a cursor). This captures a fresh snapshot of the index.

### Response structure

Each response includes:
- `count`: Number of vectors returned in this response
- `totalCount`: Total number of vectors in the index
- `isTruncated`: Whether there are more vectors available
- `nextCursor`: Cursor for the next page (null if no more results)
- `cursorExpirationTimestamp`: Timestamp of when the cursor expires
- `vectors`: Array of vector identifiers

### Cursor expiration

Cursors have an expiration timestamp. If a cursor expires, you'll need to start a new list-vectors request sequence to continue pagination.

## Performance considerations

Take care to have sufficient gap between consecutive requests to avoid hitting rate-limits.

## Example workflow

Here's a typical pattern for processing all vectors in an index:

```sh
# Start iteration
wrangler vectorize list-vectors my-index --count=1000

# Continue with cursor from response
wrangler vectorize list-vectors my-index --count=1000 --cursor="<cursor-from-response>"

# Repeat until no more results
```
3 changes: 2 additions & 1 deletion src/content/docs/vectorize/platform/limits.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,13 @@ The following limits apply to accounts, indexes and vectors (as specified):
| ------------------------------------------------------------- | ----------------------------------- |
| Indexes per account | 50,000 (Workers Paid) / 100 (Free) |
| Maximum dimensions per vector | 1536 dimensions, 32 bits precision |
| Precision per vector dimension | 32 bits (float32) |
| Precision per vector dimension | 32 bits (float32) |
| Maximum vector ID length | 64 bytes |
| Metadata per vector | 10KiB |
| Maximum returned results (`topK`) with values or metadata | 20 |
| Maximum returned results (`topK`) without values and metadata | 100 |
| Maximum upsert batch size (per batch) | 1000 (Workers) / 5000 (HTTP API) |
| Maximum vectors in a list-vectors page | 1000 |
| Maximum index name length | 64 bytes |
| Maximum vectors per index | 5,000,000 |
| Maximum namespaces per index | 50,000 (Workers Paid) / 1000 (Free) |
Expand Down
16 changes: 16 additions & 0 deletions src/content/docs/vectorize/reference/client-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,22 @@ const details = await env.YOUR_INDEX.describe();

Retrieves the configuration of a given index directly, including its configured `dimensions` and distance `metric`.

### List Vectors

List all vector identifiers in an index using paginated requests, returning up to 1000 vector identifiers per page.

```sh
wrangler vectorize list-vectors <index-name> [--count=<number>] [--cursor=<cursor-string>]
```

**Parameters:**

- `<index-name>` - The name of your Vectorize index
- `--count` (optional) - Number of vector IDs to return per page. Must be between 1 and 1000 (default: 100)
- `--cursor` (optional) - Pagination cursor from the previous response to continue listing from that position

For detailed guidance on pagination behavior and best practices, refer to [List vectors best practices](/vectorize/best-practices/list-vectors/).

### Create Metadata Index

Enable metadata filtering on the specified property. Limited to 10 properties.
Expand Down
15 changes: 15 additions & 0 deletions src/content/partials/workers/wrangler-commands/vectorize.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,21 @@ npx wrangler vectorize query <INDEX_NAME> [OPTIONS]
- `--filter` <Type text="string" /> <MetaInfo text="optional" />
- Filter vectors based on this metadata filter. Example: `'{ 'p1': 'abc', 'p2': { '$ne': true }, 'p3': 10, 'p4': false, 'nested.p5': 'abcd' }'`

<AnchorHeading title="`list-vectors`" slug="vectorize-list-vectors" depth={3} />

List vector identifiers in a Vectorize index in a paginated manner.

```sh
npx wrangler vectorize list-vectors <INDEX_NAME> [OPTIONS]
```

- `INDEX_NAME` <Type text="string" /> <MetaInfo text="required" />
- The name of the Vectorize index from which vector identifiers need to be listed.
- `--count` <Type text="number" /> <MetaInfo text="optional" />
- Number of vector IDs to return per page. Must be between 1 and 1000 (default: `100`).
- `--cursor` <Type text="string" /> <MetaInfo text="optional" />
- Pagination cursor from the previous response to continue listing from that position.

<AnchorHeading title="`get-vectors`" slug="vectorize-get-vectors" depth={3} />

Fetch vectors from a Vectorize index using the provided ids.
Expand Down
5 changes: 5 additions & 0 deletions src/content/release-notes/vectorize.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ link: "/vectorize/platform/changelog/"
productName: Vectorize
productLink: "/vectorize/"
entries:
- publish_date: "2025-08-25"
title: Added support for the list-vectors operation
description: |-
Vectorize now supports iteration through all the vector identifiers in an index in a paginated manner using the list-vectors operation.

- publish_date: "2024-12-20"
title: Added support for index name reuse
description: |-
Expand Down
Loading