Skip to content

Commit e468513

Browse files
yuancuyuancu
authored
Support time modifiers in search command (#4224)
* Implement absolute time range in search command Signed-off-by: Yuanchun Shen <[email protected]> Unit test search with absolute time range Signed-off-by: Yuanchun Shen <[email protected]> Rephrase timeRange and timeModifier Signed-off-by: Yuanchun Shen <[email protected]> Switch to earliest and latest udf Signed-off-by: Yuanchun Shen <[email protected]> Add a convert util Signed-off-by: Yuanchun Shen <[email protected]> Verify time correctness during coversion Signed-off-by: Yuanchun Shen <[email protected]> Fix quarter parsing bugs Signed-off-by: Yuanchun Shen <[email protected]> Fix week snap parsing Signed-off-by: Yuanchun Shen <[email protected]> Remove old implementation that translates time modifier to time filter Signed-off-by: Yuanchun Shen <[email protected]> * Fix anomalyzed test & add a todo for an ignored test Signed-off-by: Yuanchun Shen <[email protected]> * Support now() in time range Signed-off-by: Yuanchun Shen <[email protected]> * Fix time modifier explain ITs Signed-off-by: Yuanchun Shen <[email protected]> * Support unixtimestamp (second) as a time modifier value Signed-off-by: Yuanchun Shen <[email protected]> * Update docs for search command with time modifiers Signed-off-by: Yuanchun Shen <[email protected]> * Test accessing fields with name earliest and latest in search command Signed-off-by: Yuanchun Shen <[email protected]> * Update doctest in condition.rst due to the update in the implementation of earliest and latest conditions Signed-off-by: Yuanchun Shen <[email protected]> * Update PPLQueryDataAnonymizerTest.java Signed-off-by: Yuanchun Shen <[email protected]> * Update explain ITs to use yaml plan files Signed-off-by: Yuanchun Shen <[email protected]> * Update a link to OpenSearch exists Signed-off-by: Yuanchun Shen <[email protected]> * Support using timesnaps without quotes Signed-off-by: Yuanchun Shen <[email protected]> * Add a unit test for direct format - additionally rename parseRelativeTime to resolveTimeModifier Signed-off-by: Yuanchun Shen <[email protected]> * Add support to ISO 8601 date format to time modifier, as it is now widely supported in PPL Signed-off-by: Yuanchun Shen <[email protected]> * Update syntax by reusing SPANLENGTH definition Signed-off-by: Yuanchun Shen <[email protected]> * Update explain IT for search with time modifier Signed-off-by: Yuanchun Shen <[email protected]> * Add integration tests for time modifiers Signed-off-by: Yuanchun Shen <[email protected]> * Parse timestamp string with multiple parsers in a loop Signed-off-by: Yuanchun Shen <[email protected]> * Remove opensearch test dependency from core module Signed-off-by: Yuanchun Shen <[email protected]> * Fix unit tests Signed-off-by: Yuanchun Shen <[email protected]> * Minor updates to explain limitations in search doc Signed-off-by: Yuanchun Shen <[email protected]> --------- Signed-off-by: Yuanchun Shen <[email protected]>
1 parent e251834 commit e468513

File tree

35 files changed

+1664
-121
lines changed

35 files changed

+1664
-121
lines changed

‎core/src/main/java/org/opensearch/sql/calcite/plan/OpenSearchConstants.java‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ public interface OpenSearchConstants {
2020

2121
String METADATA_FIELD_ROUTING = "_routing";
2222

23+
String IMPLICIT_FIELD_TIMESTAMP = "@timestamp";
24+
2325
java.util.Map<String, ExprType> METADATAFIELD_TYPE_MAP =
2426
Map.of(
2527
METADATA_FIELD_ID, ExprCoreType.STRING,

‎core/src/main/java/org/opensearch/sql/expression/function/udf/condition/EarliestFunction.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,6 @@ public static Boolean earliest(Object... inputs) {
6666
ZonedDateTime earliest =
6767
getRelativeZonedDateTime(
6868
expression, ZonedDateTime.ofInstant(clock.instant(), clock.getZone()));
69-
return earliest.isBefore(candidateDatetime);
69+
return !earliest.isAfter(candidateDatetime);
7070
}
7171
}

‎core/src/main/java/org/opensearch/sql/expression/function/udf/condition/LatestFunction.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,6 @@ public static Boolean latest(Object... inputs) {
6666
ZonedDateTime latest =
6767
getRelativeZonedDateTime(
6868
expression, ZonedDateTime.ofInstant(clock.instant(), clock.getZone()));
69-
return latest.isAfter(candidateDatetime);
69+
return !latest.isBefore(candidateDatetime);
7070
}
7171
}

0 commit comments

Comments
 (0)