Skip to content

Clarifies behavior of using Direct IO for vector data #2501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Clarifies behavior of using Direct IO for vector data #2501

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,14 +121,18 @@ You can check the current value in `KiB` using `lsblk -o NAME,RA,MOUNTPOINT,TYPE
`blockdev` expects values in 512 byte sectors whereas `lsblk` reports values in `KiB`. As an example, to temporarily set readahead to `128KiB` for `/dev/nvme0n1`, specify `blockdev --setra 256 /dev/nvme0n1`.
::::


## Use Direct IO when the vector data does not fit in RAM

```{applies_to}
stack: preview 9.1
serverless: unavailable
```

If your indices are of type `bbq_hnsw` and your nodes don't have enough off-heap RAM to store all vector data in memory, you may see very high query latencies. Vector data includes the HNSW graph, quantized vectors, and raw float32 vectors.

In these scenarios, direct IO can significantly reduce query latency. Enable it by setting the JVM option `vector.rescoring.directio=true` on all vector search nodes in your cluster.
In these scenarios, direct IO can significantly reduce query latency.
Enable it by setting the JVM option `vector.rescoring.directio=true` on all vector search nodes in your cluster. {applies_to}`stack: preview 9.1.1`

`vector.rescoring.directio` is enabled by default. {applies_to}`stack: preview 9.1.0`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd probably put this in a note instead and use prose. "in {{stack}} 9.1.0, vector.rescoring.directio is enabled by default."


Only use this option if you're experiencing very high query latencies on indices of type `bbq_hnsw`. Otherwise, enabling direct IO may increase your query latencies.
Loading