[Docs] kNN vector rescoring for quantized vectors #118425
Conversation
|
Documentation preview: |
carlosdelest
force-pushed
the
feature/knn-vector-rescore-query-docs
branch
from
10c98d3 to
09a9156
Compare
…ased DoubleValueSource
…r of results from each shard
carlosdelest
force-pushed
the
feature/knn-vector-rescore-query-docs
branch
from
09a9156 to
8ade227
Compare
carlosdelest
added
v8.18.0
auto-backport
carlosdelest
added
Team:Docs
|
Pinging @elastic/es-docs (Team:Docs) |
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
| There are two main ways to oversample and rescore. The first is to utilize the <<rescore, rescore section>> in the `_search` request. | ||
| [discrete] | ||
| [[dense-vector-knn-search-reranking-rescore-section]] | ||
| ===== Use the `rescore_vector` section for top-level kNN search |
There was a problem hiding this comment.
The following example doesn't use rescore_vector, instead it utilizes the top-level rescore.
I am not sure we need to specifically call out ALL the ways that you can rescore. There are 3 ways now (rescore_vector, script score, and "rescore" at the top level). All have their uses, but our docs should be "normal user" prescriptive. Let's focus on rescore_vector as that is the easiest for oversampling and rescoring vectors.
There was a problem hiding this comment.
I think we should keep the existing ways of rescoring until we make rescore_vector GA.
We can group the other ways under a "other ways of doing rescoring" kind of section, but I think it's worth keeping them until rescore_vector is out of tech preview.
WDYT?
There was a problem hiding this comment.
I suppose calling these out is ok, we just need to be perfectly clear what each one means and when to use each one.
There was a problem hiding this comment.
I've set rescore_vector as the way to do it, and an additional section for the other methods, along with when you should use them, in ff7f906.
LMKWYT!
|
|
||
| [discrete] | ||
| [[dense-vector-knn-search-reranking-script-score]] | ||
| ===== Use a `script_score` query to rescore per shard | ||
|
|
||
| You can rescore per shard with the <<query-dsl-knn-query, knn query>> and <<query-dsl-script-score-query, script_score query >>. | ||
| Generally, this means that there will be more rescoring per shard, but this can increase overall recall at the cost of compute. |
There was a problem hiding this comment.
let's not bother calling out script_score. I say remove this section and focus on "rescore_vector"
| There are three main ways to oversample and rescore: | ||
|
|
||
| * <<dense-vector-knn-search-reranking-rescore-parameter>> | ||
| * <<dense-vector-knn-search-reranking-rescore-section>> | ||
| * <<dense-vector-knn-search-reranking-script-score>> |
There was a problem hiding this comment.
let's focus on rescore_vector and remove the other sections.
| + | ||
| include::{es-ref-dir}/rest-api/common-parms.asciidoc[tag=knn-similarity] | ||
|
|
||
| include::{es-ref-dir}/rest-api/common-parms.asciidoc[tag=knn-rescore-vector] |
There was a problem hiding this comment.
we also need to put it in search-api-knn in search.asciidoc
There was a problem hiding this comment.
Thanks for the catch - done in 9c00564
|
@benwtrent this is ready for review, it includes the latest changes for vector rescoring in #119835 |
💚 Backport successful
|
5 participants
Follow up to #116663.
Adds docs for vector rescoring.
Search your data / knn search section:
knn query / knn retriever parameters section: