Skip to content

[ESQL] Support datetime data type in Least and Greatest functions (#113961) #114138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 4, 2024
Merged

[ESQL] Support datetime data type in Least and Greatest functions (#113961) #114138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog/113961.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 113961
summary: "[ESQL] Support datetime data type in Least and Greatest functions"
area: ES|QL
type: bug
issues: []
18 changes: 18 additions & 0 deletions docs/reference/esql/functions/kibana/definition/greatest.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions docs/reference/esql/functions/kibana/definition/least.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions docs/reference/esql/functions/types/greatest.asciidoc
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions docs/reference/esql/functions/types/least.asciidoc
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions x-pack/plugin/esql/qa/testFixtures/src/main/resources/date.csv-spec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1172,3 +1172,20 @@ from employees
a:datetime
null
;


Least for dates
required_capability: least_greatest_for_dates
ROW a = LEAST(TO_DATETIME("1957-05-23T00:00:00Z"), TO_DATETIME("1958-02-19T00:00:00Z"));

a:datetime
1957-05-23T00:00:00
;

GREATEST for dates
required_capability: least_greatest_for_dates
ROW a = GREATEST(TO_DATETIME("1957-05-23T00:00:00Z"), TO_DATETIME("1958-02-19T00:00:00Z"));

a:datetime
1958-02-19T00:00:00
;
12 changes: 6 additions & 6 deletions x-pack/plugin/esql/qa/testFixtures/src/main/resources/meta.csv-spec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ double e()
"boolean ends_with(str:keyword|text, suffix:keyword|text)"
"double|integer|long|unsigned_long floor(number:double|integer|long|unsigned_long)"
"keyword from_base64(string:keyword|text)"
"boolean|double|integer|ip|keyword|long|text|version greatest(first:boolean|double|integer|ip|keyword|long|text|version, ?rest...:boolean|double|integer|ip|keyword|long|text|version)"
"boolean|date|double|integer|ip|keyword|long|text|version greatest(first:boolean|date|double|integer|ip|keyword|long|text|version, ?rest...:boolean|date|double|integer|ip|keyword|long|text|version)"
"ip ip_prefix(ip:ip, prefixLengthV4:integer, prefixLengthV6:integer)"
"boolean|double|integer|ip|keyword|long|text|version least(first:boolean|double|integer|ip|keyword|long|text|version, ?rest...:boolean|double|integer|ip|keyword|long|text|version)"
"boolean|date|double|integer|ip|keyword|long|text|version least(first:boolean|date|double|integer|ip|keyword|long|text|version, ?rest...:boolean|date|double|integer|ip|keyword|long|text|version)"
"keyword left(string:keyword|text, length:integer)"
"integer length(string:keyword|text)"
"integer locate(string:keyword|text, substring:keyword|text, ?start:integer)"
Expand Down Expand Up @@ -149,9 +149,9 @@ e |null |null
ends_with |[str, suffix] |["keyword|text", "keyword|text"] |[String expression. If `null`\, the function returns `null`., String expression. If `null`\, the function returns `null`.]
floor |number |"double|integer|long|unsigned_long" |Numeric expression. If `null`, the function returns `null`.
from_base64 |string |"keyword|text" |A base64 string.
greatest |first |"boolean|double|integer|ip|keyword|long|text|version" |First of the columns to evaluate.
greatest |first |"boolean|date|double|integer|ip|keyword|long|text|version" |First of the columns to evaluate.
ip_prefix |[ip, prefixLengthV4, prefixLengthV6]|[ip, integer, integer] |[IP address of type `ip` (both IPv4 and IPv6 are supported)., Prefix length for IPv4 addresses., Prefix length for IPv6 addresses.]
least |first |"boolean|double|integer|ip|keyword|long|text|version" |First of the columns to evaluate.
least |first |"boolean|date|double|integer|ip|keyword|long|text|version" |First of the columns to evaluate.
left |[string, length] |["keyword|text", integer] |[The string from which to return a substring., The number of characters to return.]
length |string |"keyword|text" |String expression. If `null`, the function returns `null`.
locate |[string, substring, start] |["keyword|text", "keyword|text", "integer"] |[An input string, A substring to locate in the input string, The start index]
Expand Down Expand Up @@ -392,9 +392,9 @@ e |double
ends_with |boolean |[false, false] |false |false
floor |"double|integer|long|unsigned_long" |false |false |false
from_base64 |keyword |false |false |false
greatest |"boolean|double|integer|ip|keyword|long|text|version" |false |true |false
greatest |"boolean|date|double|integer|ip|keyword|long|text|version" |false |true |false
ip_prefix |ip |[false, false, false] |false |false
least |"boolean|double|integer|ip|keyword|long|text|version" |false |true |false
least |"boolean|date|double|integer|ip|keyword|long|text|version" |false |true |false
left |keyword |[false, false] |false |false
length |integer |false |false |false
locate |integer |[false, false, true] |false |false
Expand Down
5 changes: 5 additions & 0 deletions x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,11 @@ public enum Cap {
*/
FIXED_PUSHDOWN_PAST_PROJECT,

/**
* Support for datetime in least and greatest functions
*/
LEAST_GREATEST_FOR_DATES,

/**
* Changed error messages for fields with conflicting types in different indices.
*/
Expand Down
8 changes: 4 additions & 4 deletions ...in/java/org/elasticsearch/xpack/esql/expression/function/scalar/conditional/Greatest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ public class Greatest extends EsqlScalarFunction implements OptionalArgument {
private DataType dataType;

@FunctionInfo(
returnType = { "boolean", "double", "integer", "ip", "keyword", "long", "text", "version" },
returnType = { "boolean", "date", "double", "integer", "ip", "keyword", "long", "text", "version" },
description = "Returns the maximum value from multiple columns. This is similar to <<esql-mv_max>>\n"
+ "except it is intended to run on multiple columns at once.",
note = "When run on `keyword` or `text` fields, this returns the last string in alphabetical order. "
Expand All @@ -55,12 +55,12 @@ public Greatest(
Source source,
@Param(
name = "first",
type = { "boolean", "double", "integer", "ip", "keyword", "long", "text", "version" },
type = { "boolean", "date", "double", "integer", "ip", "keyword", "long", "text", "version" },
description = "First of the columns to evaluate."
) Expression first,
@Param(
name = "rest",
type = { "boolean", "double", "integer", "ip", "keyword", "long", "text", "version" },
type = { "boolean", "date", "double", "integer", "ip", "keyword", "long", "text", "version" },
description = "The rest of the columns to evaluate.",
optional = true
) List<Expression> rest
Expand Down Expand Up @@ -153,7 +153,7 @@ public ExpressionEvaluator.Factory toEvaluator(Function<Expression, ExpressionEv
if (dataType == DataType.INTEGER) {
return new GreatestIntEvaluator.Factory(source(), factories);
}
if (dataType == DataType.LONG) {
if (dataType == DataType.LONG || dataType == DataType.DATETIME) {
return new GreatestLongEvaluator.Factory(source(), factories);
}
if (dataType == DataType.KEYWORD
Expand Down
8 changes: 4 additions & 4 deletions .../main/java/org/elasticsearch/xpack/esql/expression/function/scalar/conditional/Least.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ public class Least extends EsqlScalarFunction implements OptionalArgument {
private DataType dataType;

@FunctionInfo(
returnType = { "boolean", "double", "integer", "ip", "keyword", "long", "text", "version" },
returnType = { "boolean", "date", "double", "integer", "ip", "keyword", "long", "text", "version" },
description = "Returns the minimum value from multiple columns. "
+ "This is similar to <<esql-mv_min>> except it is intended to run on multiple columns at once.",
examples = @Example(file = "math", tag = "least")
Expand All @@ -53,12 +53,12 @@ public Least(
Source source,
@Param(
name = "first",
type = { "boolean", "double", "integer", "ip", "keyword", "long", "text", "version" },
type = { "boolean", "date", "double", "integer", "ip", "keyword", "long", "text", "version" },
description = "First of the columns to evaluate."
) Expression first,
@Param(
name = "rest",
type = { "boolean", "double", "integer", "ip", "keyword", "long", "text", "version" },
type = { "boolean", "date", "double", "integer", "ip", "keyword", "long", "text", "version" },
description = "The rest of the columns to evaluate.",
optional = true
) List<Expression> rest
Expand Down Expand Up @@ -152,7 +152,7 @@ public ExpressionEvaluator.Factory toEvaluator(Function<Expression, ExpressionEv
if (dataType == DataType.INTEGER) {
return new LeastIntEvaluator.Factory(source(), factories);
}
if (dataType == DataType.LONG) {
if (dataType == DataType.LONG || dataType == DataType.DATETIME) {
return new LeastLongEvaluator.Factory(source(), factories);
}
if (dataType == DataType.KEYWORD
Expand Down
15 changes: 15 additions & 0 deletions ...va/org/elasticsearch/xpack/esql/expression/function/scalar/conditional/GreatestTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,21 @@ public static Iterable<Object[]> parameters() {
)
)
);
suppliers.add(
new TestCaseSupplier(
"(a, b)",
List.of(DataType.DATETIME, DataType.DATETIME),
() -> new TestCaseSupplier.TestCase(
List.of(
new TestCaseSupplier.TypedData(1727877348000L, DataType.DATETIME, "a"),
new TestCaseSupplier.TypedData(1727790948000L, DataType.DATETIME, "b")
),
"GreatestLongEvaluator[values=[MvMax[field=Attribute[channel=0]], MvMax[field=Attribute[channel=1]]]]",
DataType.DATETIME,
equalTo(1727877348000L)
)
)
);
return parameterSuppliersFromTypedData(anyNullIsNull(false, suppliers));
}

Expand Down
15 changes: 15 additions & 0 deletions .../java/org/elasticsearch/xpack/esql/expression/function/scalar/conditional/LeastTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,21 @@ public static Iterable<Object[]> parameters() {
)
)
);
suppliers.add(
new TestCaseSupplier(
"(a, b)",
List.of(DataType.DATETIME, DataType.DATETIME),
() -> new TestCaseSupplier.TestCase(
List.of(
new TestCaseSupplier.TypedData(1727877348000L, DataType.DATETIME, "a"),
new TestCaseSupplier.TypedData(1727790948000L, DataType.DATETIME, "b")
),
"LeastLongEvaluator[values=[MvMin[field=Attribute[channel=0]], MvMin[field=Attribute[channel=1]]]]",
DataType.DATETIME,
equalTo(1727790948000L)
)
)
);
return parameterSuppliersFromTypedData(anyNullIsNull(false, suppliers));
}

Expand Down
Loading