-
Notifications
You must be signed in to change notification settings - Fork 25.5k
ESQL: Fix projection generation when pruning left join #135446
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Currently, in case (locally) the join key is null, the whole join is replaced with an Eval and a Project. The project however contain only reference attributes. This is insufficient, in case the attribute should point to another attribute pointing to a literal null added in the Eval. This fixes that by allowing the generated Project to contain NamedExpressions, which can then be proper Aliases.
Hi @bpintea, I've created a changelog YAML for you. |
Pinging @elastic/es-analytical-engine (Team:Analytics) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
For someone like me (who takes a PR for a spin with a sample query and dissects its internals and results) I would like to also see a concrete query in csv-spec file. You shared one with me offline and I think it wouldn't hurt to also have that one bundled in the PR, as well.
from languages_lookup_non_unique_key | eval language_code = null::integer | lookup join languages_lookup_non_unique_key on language_code
Actually... (sorry about that), I had an idea for a test and it turns out we don't have a test for that and with this PR it has a wrong behavior: Before this PR, the result is:
with this PR, the order of the columns changes (and it's wrong):
|
Indeed, due to the updated method overriding, the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes make insure that the country
field is part of the languages_lookup_non_unique_key
schema. Otherwise the CSV tests generate an index resolution without it, which leads to failed queries when using the index in FROM
.
The index was previously using similar languages
schema, which lacks the fields, but which was added in dynamically by the data. Also, the index had only been used in LOOKUP JOIN
before.
The change then also leads to updated order of the MV value ["Germany", "Austria"]
Currently, in case the join key is (locally) null, the whole join is replaced with an
Eval
and aProject
. The project however contain only reference attributes. This is insufficient, in case the attribute should point to another attribute pointing to a literalnull
added in theEval
.This fixes that by allowing the generated
Project
to containNamedExpression
s, which can then be properAlias
es.