-
Notifications
You must be signed in to change notification settings - Fork 25.7k
[WIP] Support extract_snippets function in ES|QL #132549
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…doesn't completely work yet)
…the expression evaluator
|
@carlosdelest @jimczi I plan to reach out to you next week RE: this PR. Two main questions:
|
…it multivalue aware
cb98553 to
2872969
Compare
2872969 to
5b9347c
Compare
...ain/java/org/elasticsearch/xpack/esql/expression/function/scalar/string/ExtractSnippets.java
Outdated
Show resolved
Hide resolved
...ck/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/capabilities/RewriteableAware.java
Show resolved
Hide resolved
...ck/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/capabilities/RewriteableAware.java
Show resolved
Hide resolved
ioanatia
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
when testing this, I noticed we add some weird characters at the end of the snippet.
Example:
[
"""Harry and Sally have known each other for years, and�����""",
...
],
ioanatia
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we need to ExtractSnippetTests like we have for the other functions - when we run these, we will also generate some docs files that need to be included in the PR.
we always include these generated files (some are for Kibana), even if the function is in snapshot.
x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/AnalyzerTests.java
Outdated
Show resolved
Hide resolved
...ain/java/org/elasticsearch/xpack/esql/expression/function/scalar/string/ExtractSnippets.java
Outdated
Show resolved
Hide resolved
| **Example** | ||
|
|
||
| ```{applies_to} | ||
| stack: preview 9.2.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a placeholder
🔍 Preview links for changed docs |
Provides a prototype implementation of the
EXTRACT_SNIPPETSfunction in ES|QL, for discussion on the best approaches to pursue.Here is an example of how to call this field:
You can break apart multiple values using
MV_EXPAND:EXTRACT_SNIPPETSwill work ontextandsemantic_textfields, thoughsemantic_textfields do not yet support char length so the whole chunk will be returned.Some notes on this POC: