Support ppl BETWEEN operator within Calcite#3433
Merged
LantaoJin merged 2 commits intoopensearch-project:feature/calcite-enginefrom Mar 17, 2025
Merged
Support ppl BETWEEN operator within Calcite#3433LantaoJin merged 2 commits intoopensearch-project:feature/calcite-enginefrom
LantaoJin merged 2 commits intoopensearch-project:feature/calcite-enginefrom
Conversation
Signed-off-by: Lantao Jin <ltjin@amazon.com>
LantaoJin
requested review from
GumpacG,
MaxKsyunz,
Swiddis,
YANG-DB,
Yury-Fridlyand,
acarbonetto,
anirudha,
dai-chen,
derek-ho,
forestmvey,
joshuali925,
kavithacm,
mengweieric,
noCharger,
penghuo,
ps48,
seankao-az and
ykmr1224
as code owners
Member
Author
| String expectedLogical = | ||
| "" | ||
| + "LogicalProject(EMPNO=[$0], ENAME=[$1])\n" | ||
| + " LogicalFilter(condition=[SEARCH($7, Sarg[[20..30]])])\n" |
Collaborator
There was a problem hiding this comment.
Add tests for between 20 and 30.0. Calcite has bug on Sarg: #3437, it may affect between as well.
Member
Author
There was a problem hiding this comment.
check testBetweenWithDifferentTypes2() in IT
Collaborator
There was a problem hiding this comment.
Shall we check the plan and final spark sql for that case as well? Seems we performs cast on the literal.
Member
Author
There was a problem hiding this comment.
The cast is implicit actually:
source=EMP | where DEPTNO between 20 and 30.0 | fields EMPNO, ENAME
convert to
"SELECT `EMPNO`, `ENAME` FROM `scott`.`EMP` WHERE `DEPTNO` >= 2.00E1 AND `DEPTNO` <= 3.00E1"
| JSONObject actual = | ||
| executeQuery( | ||
| String.format( | ||
| "source=%s | where not age between 30 and 39 | fields firstname, age", |
Collaborator
There was a problem hiding this comment.
Shall we add test like "where not age not between 30 and 39"? Though I think it should work well.
Member
Author
There was a problem hiding this comment.
not necessary IMO. But no harm to add it as it's simply enough.
Signed-off-by: Lantao Jin <ltjin@amazon.com>
qianheng-aws
approved these changes
Mar 17, 2025
LantaoJin
merged commit 1cefc8c
into
opensearch-project:feature/calcite-engine
9 of 13 checks passed
penghuo
pushed a commit
that referenced
this pull request
Jun 16, 2025
* Support ppl BETWEEN operation within Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * add more tests Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
BETWEENoperator within CalciteREGEXPfunctionxoroperator issue.Related Issues
Resolves #3432
Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.