-
Notifications
You must be signed in to change notification settings - Fork 25.6k
ESQL: Add more details on ENRICH vs. LOOKUP JOIN #125487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ESQL: Add more details on ENRICH vs. LOOKUP JOIN #125487
Conversation
|
Pinging @elastic/es-docs (Team:Docs) |
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
|
|
||
| * Enrichment data doesn't change frequently | ||
| * You can accept index-time overhead | ||
| * You are working with structured enrichment patterns |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't sure what this exactly means; maybe it's a bit too broad.
|
|
||
| * Your enrichment data changes frequently | ||
| * You want to avoid index-time processing | ||
| * You're working with regular indices |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was not entirely true, as lookup indices are still not perfectly the same as regular indices.
|
|
||
| The following are the current limitations with `LOOKUP JOIN` | ||
|
|
||
| * `LOOKUP JOIN` will be successful if the join field in the lookup index is a `KEYWORD` type. If the main index's join field is `TEXT` type, it must have an exact `.keyword` subfield that can be matched with the lookup index's `KEYWORD` field. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is already mentioned above and not a limitation per-se.
| ```esql | ||
| FROM <source_index> | ||
| | LOOKUP JOIN <lookup_index> ON <field_name> | ||
| ``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We use esql formatting for the syntax in the other commands.
| ```esql | ||
| FROM firewall_logs | ||
| | LOOKUP JOIN threat_list ON source.IP | ||
| | WHERE threat_level IS NOT NULL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved this below to the other examples.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! I removed 8.x labels because unfortunately we can't backport
|
💔 Backport failed
You can use sqren/backport to manually backport by running |
|
Err, I think we don't backport to 9.0, either. |
…elastic#125528) Manual backport of docs-PR elastic#125487
I think it's good to point out one more difference: there are enrich policies that match on ranges or spatial relations, which is currently not supported in LOOKUP JOIN.
Other than that, this adds a couple of minor edits (typo fixes, slight rewordings)