-
Notifications
You must be signed in to change notification settings - Fork 25.6k
[ES\QL] Text embedding function constant folding #135710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
4c8db9e
28e1185
ffb3ad4
0c9b903
dd5ec23
aaec4e1
2674d6c
3976822
4872c0c
7f9e9f4
69928fb
bf7d5df
8656357
029cee6
7e97208
5fc7461
a2822e3
2b45433
f475dbb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,91 @@ | ||
| placeholder | ||
| text_embedding using a row source operator | ||
| required_capability: text_embedding_function | ||
| required_capability: not_existing_capability | ||
| required_capability: dense_vector_field_type_released | ||
|
|
||
| // tag::embedding-eval[] | ||
| ROW input="Who is Victor Hugo?" | ||
| | EVAL embedding = TEXT_EMBEDDING("Who is Victor Hugo?", "test_dense_inference") | ||
| ; | ||
| // end::embedding-eval[] | ||
|
|
||
| input:keyword | embedding:dense_vector | ||
| Who is Victor Hugo? | [56.0, 50.0, 48.0] | ||
| ; | ||
|
|
||
|
|
||
| text_embedding using a row source operator with query build using CONCAT | ||
| required_capability: text_embedding_function | ||
| required_capability: dense_vector_field_type_released | ||
|
|
||
| ROW input="Who is Victor Hugo?" | ||
| | EVAL embedding = TEXT_EMBEDDING(CONCAT("Who is ", "Victor Hugo?"), "test_dense_inference") | ||
| ; | ||
|
|
||
| input:keyword | embedding:dense_vector | ||
| Who is Victor Hugo? | [56.0, 50.0, 48.0] | ||
| ; | ||
|
|
||
|
|
||
| text_embedding with knn on semantic_text_dense_field | ||
| required_capability: text_embedding_function | ||
| required_capability: dense_vector_field_type_released | ||
| required_capability: knn_function_v5 | ||
| required_capability: semantic_text_field_caps | ||
|
|
||
| FROM semantic_text METADATA _score | ||
| | EVAL query_embedding = TEXT_EMBEDDING("be excellent to each other", "test_dense_inference") | ||
| | WHERE KNN(semantic_text_dense_field, query_embedding) | ||
| | SORT _score DESC | ||
| | LIMIT 10 | ||
| | KEEP semantic_text_field, query_embedding | ||
| ; | ||
|
|
||
| semantic_text_field:text | query_embedding:dense_vector | ||
| be excellent to each other | [45.0, 55.0, 54.0] | ||
| live long and prosper | [45.0, 55.0, 54.0] | ||
| all we have to decide is what to do with the time that is given to us | [45.0, 55.0, 54.0] | ||
| ; | ||
|
|
||
| text_embedding with knn (inline) on semantic_text_dense_field | ||
| required_capability: text_embedding_function | ||
| required_capability: dense_vector_field_type_released | ||
| required_capability: knn_function_v5 | ||
| required_capability: semantic_text_field_caps | ||
|
|
||
| FROM semantic_text METADATA _score | ||
| | WHERE KNN(semantic_text_dense_field, TEXT_EMBEDDING("be excellent to each other", "test_dense_inference")) | ||
| | SORT _score DESC | ||
| | LIMIT 10 | ||
| | KEEP semantic_text_field | ||
| ; | ||
|
|
||
| semantic_text_field:text | ||
| be excellent to each other | ||
| live long and prosper | ||
| all we have to decide is what to do with the time that is given to us | ||
| ; | ||
|
|
||
|
|
||
| text_embedding with multiple knn queries in fork | ||
| required_capability: text_embedding_function | ||
| required_capability: dense_vector_field_type_released | ||
| required_capability: knn_function_v5 | ||
| required_capability: fork_v9 | ||
| required_capability: semantic_text_field_caps | ||
|
|
||
| FROM semantic_text METADATA _score | ||
| | FORK (EVAL query_embedding = TEXT_EMBEDDING("be excellent to each other", "test_dense_inference") | WHERE KNN(semantic_text_dense_field, query_embedding)) | ||
| (EVAL query_embedding = TEXT_EMBEDDING("live long and prosper", "test_dense_inference") | WHERE KNN(semantic_text_dense_field, query_embedding)) | ||
| | SORT _score DESC, _fork ASC | ||
| | LIMIT 10 | ||
| | KEEP semantic_text_field, query_embedding, _fork | ||
| ; | ||
|
|
||
| semantic_text_field:text | query_embedding:dense_vector | _fork:keyword | ||
| be excellent to each other | [45.0, 55.0, 54.0] | fork1 | ||
| live long and prosper | [50.0, 57.0, 56.0] | fork2 | ||
| live long and prosper | [45.0, 55.0, 54.0] | fork1 | ||
| be excellent to each other | [50.0, 57.0, 56.0] | fork2 | ||
| all we have to decide is what to do with the time that is given to us | [45.0, 55.0, 54.0] | fork1 | ||
| all we have to decide is what to do with the time that is given to us | [50.0, 57.0, 56.0] | fork2 | ||
| ; | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we get a test with multiple
text_embeddingcalls with different query strings?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a CSV test with FORK that should be covering your ask:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
plus it tests inference function in the context of fork