Skip to content

Commit 5523932

Browse files
yuancuyuancu
authored
Support chart command in PPL (#4579)
* WIP: Make poc implementation for chart command Signed-off-by: Yuanchun Shen <[email protected]> * Support param useother and otherstr Signed-off-by: Yuanchun Shen <[email protected]> * Support usenull and nullstr (when both row split and col split present) Signed-off-by: Yuanchun Shen <[email protected]> * Append a final aggregation to merge OTHER categories Signed-off-by: Yuanchun Shen <[email protected]> * Handle common agg functions for OTHER category for timechart Signed-off-by: Yuanchun Shen <[email protected]> # Conflicts: # core/src/main/java/org/opensearch/sql/calcite/CalciteRelNodeVisitor.java # integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteExplainIT.java * Fix timechart IT Signed-off-by: Yuanchun Shen <[email protected]> * Sort earliest results with asc order Signed-off-by: Yuanchun Shen <[email protected]> * Support non-string fields as column split Signed-off-by: Yuanchun Shen <[email protected]> * Fix min/earliest order & fix non-accumulative agg for chart Signed-off-by: Yuanchun Shen <[email protected]> * Hint non-null in aggregateWithTrimming Signed-off-by: Yuanchun Shen <[email protected]> * Add integration tests for chart command Signed-off-by: Yuanchun Shen <[email protected]> * Add unit tests Signed-off-by: Yuanchun Shen <[email protected]> * Add doc for chart command Signed-off-by: Yuanchun Shen <[email protected]> * Prompt users that multiple agg is not supported Signed-off-by: Yuanchun Shen <[email protected]> * Add explain ITs Signed-off-by: Yuanchun Shen <[email protected]> * Remove unimplemented support for multiple aggregations in chart command Signed-off-by: Yuanchun Shen <[email protected]> * Add unit tests for chart command Signed-off-by: Yuanchun Shen <[email protected]> * Remove irrelevant yaml test Signed-off-by: Yuanchun Shen <[email protected]> * Tweak chart.rst Signed-off-by: Yuanchun Shen <[email protected]> * Swap the order of chart output to ensure metrics come last Signed-off-by: Yuanchun Shen <[email protected]> * Filter rows without col split when calculate grand total Signed-off-by: Yuanchun Shen <[email protected]> * Chores: tweak code order Signed-off-by: Yuanchun Shen <[email protected]> * Add anonymize test to chart command Signed-off-by: Yuanchun Shen <[email protected]> * Change grammart from limit=top 10 to limit=top10 Signed-off-by: Yuanchun Shen <[email protected]> * Update chart doc Signed-off-by: Yuanchun Shen <[email protected]> * Rename __row_number__ for chart to _row_number_chart_ Signed-off-by: Yuanchun Shen <[email protected]> * Sort by row and col splits on top of chart results Signed-off-by: Yuanchun Shen <[email protected]> * Ignore rows without a row split in chart command Signed-off-by: Yuanchun Shen <[email protected]> * Keep categories with max summed values when top k is set Signed-off-by: Yuanchun Shen <[email protected]> * Simplify toAddHintsOnAggregate condition Signed-off-by: Yuanchun Shen <[email protected]> * Chores: eliminate unnecessary variables Signed-off-by: Yuanchun Shen <[email protected]> * Apply a non-null filter on fields referred by aggregations Signed-off-by: Yuanchun Shen <[email protected]> * Fix chart plans Signed-off-by: Yuanchun Shen <[email protected]> * Get rid of record class Signed-off-by: Yuanchun Shen <[email protected]> * Move ranking by column split to a helper function Signed-off-by: Yuanchun Shen <[email protected]> --------- Signed-off-by: Yuanchun Shen <[email protected]>
1 parent f483247 commit 5523932

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+2335
-180
lines changed

‎core/src/main/java/org/opensearch/sql/analysis/Analyzer.java‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@
6161
import org.opensearch.sql.ast.tree.Append;
6262
import org.opensearch.sql.ast.tree.AppendCol;
6363
import org.opensearch.sql.ast.tree.Bin;
64+
import org.opensearch.sql.ast.tree.Chart;
6465
import org.opensearch.sql.ast.tree.CloseCursor;
6566
import org.opensearch.sql.ast.tree.Dedupe;
6667
import org.opensearch.sql.ast.tree.Eval;
@@ -769,6 +770,11 @@ public LogicalPlan visitSpath(SPath node, AnalysisContext context) {
769770
throw getOnlyForCalciteException("Spath");
770771
}
771772

773+
@Override
774+
public LogicalPlan visitChart(Chart node, AnalysisContext context) {
775+
throw getOnlyForCalciteException("Chart");
776+
}
777+
772778
@Override
773779
public LogicalPlan visitTimechart(Timechart node, AnalysisContext context) {
774780
throw getOnlyForCalciteException("Timechart");

‎core/src/main/java/org/opensearch/sql/ast/AbstractNodeVisitor.java‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
import org.opensearch.sql.ast.tree.Append;
5050
import org.opensearch.sql.ast.tree.AppendCol;
5151
import org.opensearch.sql.ast.tree.Bin;
52+
import org.opensearch.sql.ast.tree.Chart;
5253
import org.opensearch.sql.ast.tree.CloseCursor;
5354
import org.opensearch.sql.ast.tree.Dedupe;
5455
import org.opensearch.sql.ast.tree.Eval;
@@ -275,6 +276,10 @@ public T visitReverse(Reverse node, C context) {
275276
return visitChildren(node, context);
276277
}
277278

279+
public T visitChart(Chart node, C context) {
280+
return visitChildren(node, context);
281+
}
282+
278283
public T visitTimechart(Timechart node, C context) {
279284
return visitChildren(node, context);
280285
}

‎core/src/main/java/org/opensearch/sql/ast/tree/Chart.java‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
/*
2+
* Copyright OpenSearch Contributors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package org.opensearch.sql.ast.tree;
7+
8+
import com.google.common.collect.ImmutableList;
9+
import java.util.List;
10+
import lombok.AllArgsConstructor;
11+
import lombok.EqualsAndHashCode;
12+
import lombok.Getter;
13+
import lombok.ToString;
14+
import org.opensearch.sql.ast.AbstractNodeVisitor;
15+
import org.opensearch.sql.ast.dsl.AstDSL;
16+
import org.opensearch.sql.ast.expression.Argument;
17+
import org.opensearch.sql.ast.expression.Literal;
18+
import org.opensearch.sql.ast.expression.UnresolvedExpression;
19+
20+
/** AST node represent chart command. */
21+
@Getter
22+
@ToString
23+
@EqualsAndHashCode(callSuper = false)
24+
@AllArgsConstructor
25+
@lombok.Builder(toBuilder = true)
26+
public class Chart extends UnresolvedPlan {
27+
public static final Literal DEFAULT_USE_OTHER = Literal.TRUE;
28+
public static final Literal DEFAULT_OTHER_STR = AstDSL.stringLiteral("OTHER");
29+
public static final Literal DEFAULT_LIMIT = AstDSL.intLiteral(10);
30+
public static final Literal DEFAULT_USE_NULL = Literal.TRUE;
31+
public static final Literal DEFAULT_NULL_STR = AstDSL.stringLiteral("NULL");
32+
public static final Literal DEFAULT_TOP = Literal.TRUE;
33+
34+
private UnresolvedPlan child;
35+
private UnresolvedExpression rowSplit;
36+
private UnresolvedExpression columnSplit;
37+
private UnresolvedExpression aggregationFunction;
38+
private List<Argument> arguments;
39+
40+
@Override
41+
public UnresolvedPlan attach(UnresolvedPlan child) {
42+
this.child = child;
43+
return this;
44+
}
45+
46+
@Override
47+
public List<UnresolvedPlan> getChild() {
48+
return this.child == null ? ImmutableList.of() : ImmutableList.of(this.child);
49+
}
50+
51+
@Override
52+
public <T, C> T accept(AbstractNodeVisitor<T, C> nodeVisitor, C context) {
53+
return nodeVisitor.visitChart(this, context);
54+
}
55+
}

0 commit comments

Comments
 (0)