Skip to content

Conversation

benwtrent
Copy link
Member

Adds clarification for vector preloading, what extension is to what storage kind, and that quantized vectors are stored in separate files allowing for individual preload.

closes: #116273

@benwtrent benwtrent added >docs General docs changes auto-backport Automatically create backport pull requests when merged :Search Relevance/Vectors Vector search v9.0.0 v8.16.1 v8.17.0 labels Nov 8, 2024
Copy link
Contributor

github-actions bot commented Nov 8, 2024

Documentation preview:

@elasticsearchmachine elasticsearchmachine added Team:Docs Meta label for docs team Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Nov 8, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

* `vec` for all non-quantized vector values. This includes all element types: `float`, `byte`, and `bit`.
* `veq` for quantized vectors indexed with <<dense-vector-quantization,`quantization`>>: `int4` or `int8`
* `veb` for binary vectors indexed with <<dense-vector-quantization,`quantization`>>: `bbq`
* `vem`, `vemf`, `vemq`, and `vemb` for metadata, usually small and not a concern for preloading
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for leaving vex out of this list, or was it just overlooked?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it was indeed overlooked, let me add that.

Copy link
Contributor

@pmpailis pmpailis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ; only a minor comment on a field type

@benwtrent
Copy link
Member Author

@elasticmachine update branch

@benwtrent benwtrent added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Nov 11, 2024
@elasticsearchmachine elasticsearchmachine merged commit 89467b2 into elastic:main Nov 11, 2024
5 checks passed
@benwtrent benwtrent deleted the docs/better-explain-vector-preloading branch November 11, 2024 21:31
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.16
8.x

benwtrent added a commit to benwtrent/elasticsearch that referenced this pull request Nov 11, 2024
Adds clarification for vector preloading, what extension is to what
storage kind, and that quantized vectors are stored in separate files
allowing for individual preload. 

closes: elastic#116273
benwtrent added a commit to benwtrent/elasticsearch that referenced this pull request Nov 11, 2024
Adds clarification for vector preloading, what extension is to what
storage kind, and that quantized vectors are stored in separate files
allowing for individual preload. 

closes: elastic#116273
elasticsearchmachine pushed a commit that referenced this pull request Nov 11, 2024
Adds clarification for vector preloading, what extension is to what
storage kind, and that quantized vectors are stored in separate files
allowing for individual preload. 

closes: #116273
elasticsearchmachine pushed a commit that referenced this pull request Nov 11, 2024
Adds clarification for vector preloading, what extension is to what
storage kind, and that quantized vectors are stored in separate files
allowing for individual preload. 

closes: #116273
jozala pushed a commit that referenced this pull request Nov 13, 2024
Adds clarification for vector preloading, what extension is to what
storage kind, and that quantized vectors are stored in separate files
allowing for individual preload. 

closes: #116273
afoucret pushed a commit to afoucret/elasticsearch that referenced this pull request Nov 14, 2024
Adds clarification for vector preloading, what extension is to what
storage kind, and that quantized vectors are stored in separate files
allowing for individual preload. 

closes: elastic#116273
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this pull request Nov 28, 2024
Adds clarification for vector preloading, what extension is to what
storage kind, and that quantized vectors are stored in separate files
allowing for individual preload. 

closes: elastic#116273
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >docs General docs changes :Search Relevance/Vectors Vector search Team:Docs Meta label for docs team Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v8.16.1 v8.17.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clarify vector preloading docs

4 participants