-
Notifications
You must be signed in to change notification settings - Fork 25.6k
Adds maxSim
functions for multi_dense_vector fields
#116993
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds maxSim
functions for multi_dense_vector fields
#116993
Conversation
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
@joshdevins you might be interested in this. Anything significant missing? |
server/src/main/java/org/elasticsearch/script/field/vectors/BitMultiDenseVector.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's amazing @benwtrent !
I left a minor suggestion and a question, LGTM otherwise.
server/src/main/java/org/elasticsearch/script/field/vectors/BitMultiDenseVector.java
Outdated
Show resolved
Hide resolved
|
||
@Override | ||
public Iterator<byte[]> copy() { | ||
return new ByteVectorIterator(vectorValues, buffer, size); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it ok to reuse the buffer instead of creating a new one for each copy?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, doing the following:
def field = field('vector').get();
def vvs = field.getVectors();
def vvs2 = field.getVectors();
def v1 = vvs.next();
def otherV1 = vvs2.next();
return v1[1];",
Will actually cause the second vector in vvs2 twice. Meaning, calling getVectors()
without having a copy underneath are actually sharing iterators & buffer meaning v1
s value is transformed with the vvs2.next()
Copying the buffer to keep users from shooting themselves in the foot with two iterators seems like a simple thing to do. This would be only for expert users regardless.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In short, yes @jimczi reusing the buffer is a bad idea. Fixed that and now copy will create a new buffer. This way all iterators have their own buffer, but for typical maxsim scoring, we can reuse the buffer there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the delay in commenting. I agree that we should not re-use the buffer given that while it's probably safe for expert users, I worry about future code changes that affect the re-usability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Super minor comments about error messages. I think people could be confused at times since we're dealing with 2D tenors. I continue to wish we had tensor types in Elasticsearch to avoid some of these kinds of issues from arising in the first place.
server/src/main/java/org/elasticsearch/script/MultiVectorScoreScriptUtils.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/script/MultiVectorScoreScriptUtils.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/script/MultiVectorScoreScriptUtils.java
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/script/MultiVectorScoreScriptUtils.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/script/MultiVectorScoreScriptUtils.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/script/MultiVectorScoreScriptUtils.java
Outdated
Show resolved
Hide resolved
...ng-painless/src/main/resources/org/elasticsearch/painless/org.elasticsearch.script.score.txt
Show resolved
Hide resolved
...ss/src/yamlRestTest/resources/rest-api-spec/test/painless/141_multi_dense_vector_max_sim.yml
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Ben, great addition
@elasticmachine update branch |
💚 Backport successful
|
This adds `maxSim` functions, specifically dotProduct and InvHamming. Why these two you might ask? Well, they are the best approximations of whats possible with Col* late interaction type models. Effectively, you want a similarity metric where "greater == better". Regular `hamming` isn't exactly that, but inverting that (just like our `element_type: bit` index for dense_vectors), is a nice approximation with bit vectors and multi-vector scoring. Then, of course, dotProduct is another usage. We will allow dot-product between like elements (bytes -> bytes, floats -> floats) and of course, allow `floats -> bit`, where the stored `bit` elements are applied as a "mask" over the float queries. This allows for some nice asymmetric interactions. This is all behind a feature flag, and I need to write a mountain of docs in a separate PR.
This adds `maxSim` functions, specifically dotProduct and InvHamming. Why these two you might ask? Well, they are the best approximations of whats possible with Col* late interaction type models. Effectively, you want a similarity metric where "greater == better". Regular `hamming` isn't exactly that, but inverting that (just like our `element_type: bit` index for dense_vectors), is a nice approximation with bit vectors and multi-vector scoring. Then, of course, dotProduct is another usage. We will allow dot-product between like elements (bytes -> bytes, floats -> floats) and of course, allow `floats -> bit`, where the stored `bit` elements are applied as a "mask" over the float queries. This allows for some nice asymmetric interactions. This is all behind a feature flag, and I need to write a mountain of docs in a separate PR.
) This adds `maxSim` functions, specifically dotProduct and InvHamming. Why these two you might ask? Well, they are the best approximations of whats possible with Col* late interaction type models. Effectively, you want a similarity metric where "greater == better". Regular `hamming` isn't exactly that, but inverting that (just like our `element_type: bit` index for dense_vectors), is a nice approximation with bit vectors and multi-vector scoring. Then, of course, dotProduct is another usage. We will allow dot-product between like elements (bytes -> bytes, floats -> floats) and of course, allow `floats -> bit`, where the stored `bit` elements are applied as a "mask" over the float queries. This allows for some nice asymmetric interactions. This is all behind a feature flag, and I need to write a mountain of docs in a separate PR.
This adds `maxSim` functions, specifically dotProduct and InvHamming. Why these two you might ask? Well, they are the best approximations of whats possible with Col* late interaction type models. Effectively, you want a similarity metric where "greater == better". Regular `hamming` isn't exactly that, but inverting that (just like our `element_type: bit` index for dense_vectors), is a nice approximation with bit vectors and multi-vector scoring. Then, of course, dotProduct is another usage. We will allow dot-product between like elements (bytes -> bytes, floats -> floats) and of course, allow `floats -> bit`, where the stored `bit` elements are applied as a "mask" over the float queries. This allows for some nice asymmetric interactions. This is all behind a feature flag, and I need to write a mountain of docs in a separate PR.
Hi guys, good work on this. Do you know in what version this will be released? Keen to try it out. |
@webeng 8.18.0, I am not sure the exact date for that release. Its in https://www.elastic.co/cloud/serverless right now. Note, the mapping is now called |
This adds
maxSim
functions, specifically dotProduct and InvHamming. Why these two you might ask? Well, they are the best approximations of whats possible with Col* late interaction type models. Effectively, you want a similarity metric where "greater == better". Regularhamming
isn't exactly that, but inverting that (just like ourelement_type: bit
index for dense_vectors), is a nice approximation with bit vectors and multi-vector scoring.Then, of course, dotProduct is another usage. We will allow dot-product between like elements (bytes -> bytes, floats -> floats) and of course, allow
floats -> bit
, where the storedbit
elements are applied as a "mask" over the float queries. This allows for some nice asymmetric interactions.This is all behind a feature flag, and I need to write a mountain of docs in a separate PR.