Add datetime functions by xinyual · Pull Request #3473 · opensearch-project/sql

xinyual · 2025-03-24T12:00:50Z

Description

We have several tasks in this PR:

Add datetime functions.
Add IT for these functions
Add approximate percentile functions and IT
Add comparison between time/datetime/date

For this PR,
we miss

Validation for argument

Related Issues

Resolves #[Issue number to be closed when this PR is merged]
#3400

I have tested the nonFallbackITs in the #3400 locally. There are some issues:
NonFallbackCalciteNowLikeFunctionIT: pass all
NonFallbackCalciteDateTimeFunctionIT: pass all

NonFallbackCalciteDateTimeImplementationIT: pass
NonFallbackCalciteDateTimeComparisonIT: pass all
NonFallbackCalciteConvertTZFunctionIT: pass

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: xinyual <xinyual@amazon.com>

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Signed-off-by: xinyual <xinyual@amazon.com>

Signed-off-by: Xinyuan Lu <xinyual@amazon.com>

Map opensearch math functions to calcite implementations

Signed-off-by: xinyual <xinyual@amazon.com>

…StringFunctions

Signed-off-by: xinyual <xinyual@amazon.com>

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

LantaoJin · 2025-04-01T09:46:10Z

core/src/main/java/org/opensearch/sql/calcite/CalciteRexNodeVisitor.java

    }
  }

+  private SpanUnit intervalUnitToSpanUnit(IntervalUnit unit) {


Reuse this https://github.com/opensearch-project/sql/blob/main/core/src/main/java/org/opensearch/sql/calcite/utils/PlanUtils.java#L13

Already done.

LantaoJin · 2025-04-01T09:50:00Z

...rc/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/PostprocessDateToStringFunction.java

@@ -0,0 +1,25 @@
+package org.opensearch.sql.calcite.udf.datetimeUDF;


copyright. please check all new added files.

IDE: settings -> Editor -> Copyright -> Copyright Profile -> +

LantaoJin · 2025-04-01T09:53:36Z

core/src/main/java/org/opensearch/sql/calcite/CalciteRexNodeVisitor.java


+  private RexNode transferCompareForDateRelated(
+      RexNode candidate, CalcitePlanContext context, boolean wrapper) {
+    if (wrapper) {


wrapper -> whetherCompareByTime

do we still need this DateToString call?

Do you mean PostprocessDateToStringFunction?

LantaoJin · 2025-04-01T09:57:58Z

core/src/main/java/org/opensearch/sql/calcite/CalciteRexNodeVisitor.java


  @Override
  public RexNode visitFunction(Function node, CalcitePlanContext context) {
+    // Re-org the code structure


remove this comment

LantaoJin · 2025-04-01T09:58:42Z

core/src/main/java/org/opensearch/sql/calcite/udf/UserDefinedAggFunction.java

 package org.opensearch.sql.calcite.udf;

+/**
+ * TO DO. support init with constant arguments https://github.com/opensearch-project/sql/issues/3490


TODO, not TO DO

LantaoJin · 2025-04-01T10:00:37Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/PostprocessForUDTFunction.java

@@ -0,0 +1,37 @@
+package org.opensearch.sql.calcite.udf.datetimeUDF;


LantaoJin · 2025-04-01T10:01:32Z

core/src/main/java/org/opensearch/sql/calcite/utils/BuiltinFunctionUtils.java

 import org.opensearch.sql.calcite.udf.textUDF.LocateFunction;
 import org.opensearch.sql.calcite.udf.textUDF.ReplaceFunction;

+/** TO DO: We need to refactor code to make all */


LantaoJin · 2025-04-01T10:02:22Z

core/src/main/java/org/opensearch/sql/calcite/utils/BuiltinFunctionUtils.java

@@ -204,6 +331,82 @@ static SqlOperator translate(String op) {
        return TransferUserDefinedFunction(
            TypeOfFunction.class, "typeof", ReturnTypes.VARCHAR_2000_NULLABLE);
        // TODO Add more, ref RexImpTable


move this line before default:

LantaoJin · 2025-04-01T10:05:25Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/DateFunction.java

+    // try {
+    //  return exprDate(transferInputToExprTimestampValue(args[0], (SqlTypeName) args[1], restored))
+    //      .valueForCalcite();
+    // } catch (SemanticCheckException e) {
+    //  return null;
+    // }


remove them

LantaoJin · 2025-04-01T10:05:43Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/TimeAddSubFunction.java

+    // Instant base = InstantUtils.convertToInstant(argBase, baseType, false);
+    // Instant interval = InstantUtils.convertToInstant(argInterval, argIntervalType, false);
+    // LocalTime time = interval.atZone(ZoneOffset.UTC).toLocalTime();
+    // Duration duration = Duration.between(LocalTime.MIN, time);
+
+    // Instant newInstant = DateTimeApplyUtils.applyInterval(base, duration, isAdd);


qianheng-aws · 2025-04-01T09:43:07Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/DateAddSubFunction.java

+    boolean isAdd = (Boolean) args[4];
+    SqlTypeName returnSqlType = (SqlTypeName) args[5];
+    ExprValue base = transferInputToExprValue(argBase, sqlTypeName);
+    // Instant base = InstantUtils.convertToInstant(argBase, sqlTypeName, false);


qianheng-aws · 2025-04-01T10:10:58Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/DateAddSubFunction.java

+      return null;
+    }
+
+    if (UserDefinedFunctionUtils.containsNull(args)) {


Already remove it.

qianheng-aws · 2025-04-01T10:12:38Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/DateAddSubFunction.java

+    TimeUnit unit = (TimeUnit) args[0];
+    long interval = ((Number) args[1]).longValue();
+    Object argBase = args[2];
+    SqlTypeName sqlTypeName = (SqlTypeName) args[3];


Why using SqlTypeName here? I think sql type is a subset of all types we use and date, time, timestamp of Sql type is no longer used by us after introducing UDT

For our current UDF, all time/timestamp/time input would be the UDT. So we directly transfer the type to corresponding sqltypetime that we can call fromObjectValue easily. See function transferDateRelatedTimeName

These UDT's real SqlType is VARCHAR, the more appropriate way is pass in the RelDataType here and get the correct exprtype by using ExprUDT.getExprCoreType.

It works here because we have a incorrect mapping in convertSqlTypeNameToExprType(the mapping of time,timestamp,date will never be used since they are blocked now by the previous logic of handling UDT), and you pass in SqlTypeName.TIMESTAMP directly from the place where you call this funciton.

qianheng-aws · 2025-04-01T10:14:21Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/DateAddSubFunction.java

+    return opBinding -> {
+      RelDataType operandType0 = opBinding.getOperandType(6);
+      SqlTypeName typeName = operandType0.getSqlTypeName();
+      if (typeName == SqlTypeName.TIMESTAMP) {


In which case will we use SqlTypeName.TIMESTAMP?

Same as above.

qianheng-aws · 2025-04-01T10:37:54Z

core/src/main/java/org/opensearch/sql/calcite/udf/datetimeUDF/DateAddSubFunction.java

+    SqlTypeName returnSqlType = (SqlTypeName) args[5];
+    ExprValue base = transferInputToExprValue(argBase, sqlTypeName);
+    // Instant base = InstantUtils.convertToInstant(argBase, sqlTypeName, false);
+    FunctionProperties restored = restoreFunctionProperties(args[args.length - 1]);


This properties should be set when constructing the function and I think it supports pass in FunctionProperties directly, please add a TODO here to make change in the future.

Got it. Add in function restoreFunctionProperties

Signed-off-by: xinyual <xinyual@amazon.com>

qianheng-aws · 2025-04-02T08:45:53Z

...t/java/org/opensearch/sql/calcite/standalone/CalcitePPLBuiltinDatetimeFunctionInvalidIT.java

+
+  @Test
+  public void testYearWeekInvalid() {
+    assertThrows(


Please add assert for error message as well.

Already add.

LantaoJin · 2025-04-02T09:33:41Z

...src/test/java/org/opensearch/sql/calcite/standalone/CalcitePPLDateTimeBuiltinFunctionIT.java

   * date value given the day number N. DATETIME: (TIMESTAMP, STRING) -> TIMESTAMP (TIMESTAMP) ->
   * TIMESTAMP Converts the datetime to a new timezone
   */
+  @Ignore


why this test flaky?

Already fix the bug and enable IT.

Signed-off-by: xinyual <xinyual@amazon.com>

…cal digits) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Fix date / time formatting

Signed-off-by: xinyual <xinyual@amazon.com>

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Reuse PlanUtils.intervalUnitToSpanUnit

Signed-off-by: xinyual <xinyual@amazon.com>

LantaoJin

LGTM, thanks @xinyual and @yuancu

* add condition and string func Signed-off-by: xinyual <xinyual@amazon.com> * add atan Signed-off-by: xinyual <xinyual@amazon.com> * [WIP] map opensearch math functions to calcite implementations Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * add transfer for log Signed-off-by: xinyual <xinyual@amazon.com> * fix log and atan Signed-off-by: xinyual <xinyual@amazon.com> * add udf and udaf percentile and Mod Signed-off-by: xinyual <xinyual@amazon.com> * add condition udf and take agg Signed-off-by: xinyual <xinyual@amazon.com> * add unified test framework Signed-off-by: xinyual <xinyual@amazon.com> * fix UT/IT Signed-off-by: xinyual <xinyual@amazon.com> * fix UT/IT Signed-off-by: xinyual <xinyual@amazon.com> * remove useless code change Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * add license and move it Signed-off-by: xinyual <xinyual@amazon.com> * remove useless change Signed-off-by: xinyual <xinyual@amazon.com> * add time functions Signed-off-by: xinyual <xinyual@amazon.com> * fix date/utc_data/timestamp Signed-off-by: xinyual <xinyual@amazon.com> * add unix_timestamp(half) Signed-off-by: xinyual <xinyual@amazon.com> * continue to fix unixtimestamp Signed-off-by: xinyual <xinyual@amazon.com> * fix unixtimestamp Signed-off-by: xinyual <xinyual@amazon.com> * add timestamp Signed-off-by: xinyual <xinyual@amazon.com> * fix timestamp Signed-off-by: xinyual <xinyual@amazon.com> * Implement DATEADD function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement DATE_SUB function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * WIP: Implement TIMEADD Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement ADDTIME Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement DAY_OF_WEEK and DAY_OF_YEAR Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement DATEDIFF Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Leverage org.opensearch.sql.utils.DateTimeFormatters for datetime parsing Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Make datetime utils interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement DATE_FORMAT (can't handle nanoseconds yet) TODOs: - support nanoseconds - correct time to support date as arguments Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix rebase Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix TIME function - Additionally apply spotless Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement EXTRACT by reusing OS PPL Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement CONVERT_TZ. Cannot handle null return at the moment. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * Implement DATETIME - TODO: handle null returns Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement FROM_DAYS Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement GET_FORMAT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement MAKETIME - TODO: fix returned time representation Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * add to days Signed-off-by: xinyual <xinyual@amazon.com> * Fix null returns for CONVERT_TZ and DATETIME by manually casting returns to long in UDF Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * add time functions Signed-off-by: xinyual <xinyual@amazon.com> * Implement PERIOD_ADD and PERIOD_DIFF Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement STR_TO_DATE Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add license information to datetime UDFs Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement SUBTIME Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * add date functions Signed-off-by: xinyual <xinyual@amazon.com> * Reimplement WEEK and WEEK_OF_YEAR Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Map CURRENT_TIME to SqlStdOperatorTable.CURRENT_TIME Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement MINUTE_OF_DAY Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix String.format locale to US for exprMakeTime Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * finish all datetime functions Signed-off-by: xinyual <xinyual@amazon.com> * fix implementation Signed-off-by: xinyual <xinyual@amazon.com> * add it for time/timestamp/date Signed-off-by: xinyual <xinyual@amazon.com> * add relative time Signed-off-by: xinyual <xinyual@amazon.com> * add relative Signed-off-by: xinyual <xinyual@amazon.com> * fix year bug Signed-off-by: xinyual <xinyual@amazon.com> * fix config Signed-off-by: xinyual <xinyual@amazon.com> * add several IT Signed-off-by: xinyual <xinyual@amazon.com> * Correct DATE_ADD and DATE_SUB implementations Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix date and time fields reading Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * fix from unix timestamp Signed-off-by: xinyual <xinyual@amazon.com> * add to_days Signed-off-by: xinyual <xinyual@amazon.com> * Add date manipulation ITs Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * fix sec to time Signed-off-by: xinyual <xinyual@amazon.com> * fix millisecond Signed-off-by: xinyual <xinyual@amazon.com> * fix time expression Signed-off-by: xinyual <xinyual@amazon.com> * fix nullable problem Signed-off-by: xinyual <xinyual@amazon.com> * merge from main Signed-off-by: xinyual <xinyual@amazon.com> * Correct DATE_PART related functions (HOUR, MINUTE, etc) to correctly handle string inputs Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement MAKEDATE Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Correct periodNameFunction to correctly parse datetime string and to always return English(US)-styled day & month names Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * add IT Signed-off-by: xinyual <xinyual@amazon.com> * Support QUARTER function and add ITs Signed-off-by: Songkan Tang <songkant@amazon.com> * Add SECOND function ITs Signed-off-by: Songkan Tang <songkant@amazon.com> * Add SECOND_OF_MINUTE function ITs Signed-off-by: Songkan Tang <songkant@amazon.com> * [Calcite engine] TimeStamp UDT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add convert_tz ITs Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor fix of get_format function and add ITs Signed-off-by: Songkan Tang <songkant@amazon.com> * Correct DATETIME function to accept timestamp argument Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add extract function with supported ITs Signed-off-by: Songkan Tang <songkant@amazon.com> * Correct DATEDIFF to align with V2's behavior Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Correct date part return types & Fix date format locale to English Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add date time ITs Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add ITs for SYSDATE and TIMEDIFF Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert fixing locale to allow local representation of date time Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * fix sec to time Signed-off-by: xinyual <xinyual@amazon.com> * add license Signed-off-by: xinyual <xinyual@amazon.com> * finish useless code Signed-off-by: xinyual <xinyual@amazon.com> * add IT for percentile Signed-off-by: xinyual <xinyual@amazon.com> * rename files Signed-off-by: xinyual <xinyual@amazon.com> * fix to days Signed-off-by: xinyual <xinyual@amazon.com> * fix Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * fix null for yearweek Signed-off-by: xinyual <xinyual@amazon.com> * add null IT Signed-off-by: xinyual <xinyual@amazon.com> * Add ITs for ADDTIME, ADDDATE, DATESUB, DATEADD, DATE Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Parse date / time exclusively for YEAR, QUARTER, MONTH, DAY, HOUR, MINUTE, SECOND, etc - it won't parse a time string where a date string is required Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * add IT Signed-off-by: xinyual <xinyual@amazon.com> * Support timestamp udt Signed-off-by: Heng Qian <qianheng@amazon.com> * Support timestamp udt Signed-off-by: Heng Qian <qianheng@amazon.com> * Add date time null input ITs Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * deal invalid cases Signed-off-by: xinyual <xinyual@amazon.com> * Make all udf return types nullable Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Remove nullable argument in getReturnTypeForAddOrSubDate since it's no longer necessary Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * enable nonfallback datetimes Signed-off-by: xinyual <xinyual@amazon.com> * Support date, time udt Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix date time null returns & Add null ITs Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Allow timestamp string where date or time string is required Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Remove redundant nullify of return types Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * fix percentile Signed-off-by: xinyual <xinyual@amazon.com> * Correct makeConversionCall Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement TIME UDF Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix bug caused by cache in RelDataTypeImpl Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> * inte with udt Signed-off-by: xinyual <xinyual@amazon.com> * Ignore UT with span on calcite date Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix bug Signed-off-by: Heng Qian <qianheng@amazon.com> * fix add sub time Signed-off-by: xinyual <xinyual@amazon.com> * Fix bug Signed-off-by: Heng Qian <qianheng@amazon.com> * fix IT Signed-off-by: xinyual <xinyual@amazon.com> * fix IT Signed-off-by: xinyual <xinyual@amazon.com> * fix return type for str to date Signed-off-by: xinyual <xinyual@amazon.com> * fix for subtime Signed-off-by: xinyual <xinyual@amazon.com> * add postprocessing for curtimes Signed-off-by: xinyual <xinyual@amazon.com> * fix all IT Signed-off-by: xinyual <xinyual@amazon.com> * fix format bug Signed-off-by: xinyual <xinyual@amazon.com> * fix function properties for all utc function Signed-off-by: xinyual <xinyual@amazon.com> * use fixed function properties Signed-off-by: xinyual <xinyual@amazon.com> * fix all IT Signed-off-by: xinyual <xinyual@amazon.com> * remove all useless code Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * Fix date / time parsing with microseconds - enrich ITs with microseconds Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test TIMESTAMPADD and TIMESTAMPDIFF with microseconds Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * remove uesless Signed-off-by: xinyual <xinyual@amazon.com> * add compare Signed-off-by: xinyual <xinyual@amazon.com> * revert change Signed-off-by: xinyual <xinyual@amazon.com> * fix IT Signed-off-by: xinyual <xinyual@amazon.com> * fix comparison bug Signed-off-by: xinyual <xinyual@amazon.com> * fix spotless Signed-off-by: xinyual <xinyual@amazon.com> * fix IT Signed-off-by: xinyual <xinyual@amazon.com> * fix IT Signed-off-by: xinyual <xinyual@amazon.com> * fix IT Signed-off-by: xinyual <xinyual@amazon.com> * Fix testMinuteOfHourAndMinuteOfDay IT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * remove useless code Signed-off-by: xinyual <xinyual@amazon.com> * add pushdown IT Signed-off-by: xinyual <xinyual@amazon.com> * revert useless change Signed-off-by: xinyual <xinyual@amazon.com> * format code Signed-off-by: xinyual <xinyual@amazon.com> * modify code Signed-off-by: xinyual <xinyual@amazon.com> * remove useless Signed-off-by: xinyual <xinyual@amazon.com> * remove useless Signed-off-by: xinyual <xinyual@amazon.com> * enable IT Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * ignore failed ut Signed-off-by: xinyual <xinyual@amazon.com> * use v2 implementation Signed-off-by: xinyual <xinyual@amazon.com> * fix implementation Signed-off-by: xinyual <xinyual@amazon.com> * use v2 implementation for timestamp and time Signed-off-by: xinyual <xinyual@amazon.com> * implement by v2 Signed-off-by: xinyual <xinyual@amazon.com> * fix implementation Signed-off-by: xinyual <xinyual@amazon.com> * fix date format problem Signed-off-by: xinyual <xinyual@amazon.com> * remove useless code Signed-off-by: xinyual <xinyual@amazon.com> * fix date add/sub Signed-off-by: xinyual <xinyual@amazon.com> * remove useless code Signed-off-by: xinyual <xinyual@amazon.com> * simplify code Signed-off-by: xinyual <xinyual@amazon.com> * refactor date implementation Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * add to do Signed-off-by: xinyual <xinyual@amazon.com> * add TO DO Signed-off-by: xinyual <xinyual@amazon.com> * remove test code Signed-off-by: xinyual <xinyual@amazon.com> * refactor code Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless Signed-off-by: xinyual <xinyual@amazon.com> * fix some bugs Signed-off-by: xinyual <xinyual@amazon.com> * fix IT Signed-off-by: xinyual <xinyual@amazon.com> * ignore flaky test Signed-off-by: xinyual <xinyual@amazon.com> * reformat code to totally use v2 implementation Signed-off-by: xinyual <xinyual@amazon.com> * remove useless code Signed-off-by: xinyual <xinyual@amazon.com> * remove useless code Signed-off-by: xinyual <xinyual@amazon.com> * revert change Signed-off-by: xinyual <xinyual@amazon.com> * add copy right Signed-off-by: xinyual <xinyual@amazon.com> * change compare to local date time Signed-off-by: xinyual <xinyual@amazon.com> * change name after merge Signed-off-by: xinyual <xinyual@amazon.com> * remove useless code Signed-off-by: xinyual <xinyual@amazon.com> * fix bug Signed-off-by: xinyual <xinyual@amazon.com> * ignore flaky test Signed-off-by: xinyual <xinyual@amazon.com> * modify IT message information Signed-off-by: xinyual <xinyual@amazon.com> * Fix date / time formatting (always prefer Arabic digits instead of local digits) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * apply spot Signed-off-by: xinyual <xinyual@amazon.com> * Reuse PlanUtils.intervalUnitToSpanUnit Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * revert useless code Signed-off-by: xinyual <xinyual@amazon.com> * apply spotless and revert useless change Signed-off-by: xinyual <xinyual@amazon.com> --------- Signed-off-by: xinyual <xinyual@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Co-authored-by: xinyual <xinyual@amazon.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Heng Qian <qianheng@amazon.com> Signed-off-by: xinyual <xinyual@amazon.com>

xinyual and others added 30 commits February 24, 2025 10:42

add condition and string func

7f1eba4

Signed-off-by: xinyual <xinyual@amazon.com>

add atan

86f1b9c

Signed-off-by: xinyual <xinyual@amazon.com>

[WIP] map opensearch math functions to calcite implementations

7d46cbc

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

add transfer for log

ce264dd

Signed-off-by: xinyual <xinyual@amazon.com>

Merge branch 'mapToCalciteBuiltIn' into map2calcite

06a7b65

Signed-off-by: Xinyuan Lu <xinyual@amazon.com>

Merge pull request #1 from yaunchun/map2calcite

25d7d51

Map opensearch math functions to calcite implementations

fix log and atan

6cc2452

Signed-off-by: xinyual <xinyual@amazon.com>

add udf and udaf percentile and Mod

7ff8a1c

Signed-off-by: xinyual <xinyual@amazon.com>

add condition udf and take agg

19e11d6

Signed-off-by: xinyual <xinyual@amazon.com>

add unified test framework

25867df

Signed-off-by: xinyual <xinyual@amazon.com>

Merge remote-tracking branch 'origin/feature/calcite-engine' into add…

65fd487

…StringFunctions

fix UT/IT

7274a05

Signed-off-by: xinyual <xinyual@amazon.com>

merge

6ccfd97

Signed-off-by: xinyual <xinyual@amazon.com>

fix UT/IT

f6120dc

Signed-off-by: xinyual <xinyual@amazon.com>

remove useless code change

4fa915d

Signed-off-by: xinyual <xinyual@amazon.com>

apply spotless

99b33cd

Signed-off-by: xinyual <xinyual@amazon.com>

add license and move it

0b28fab

Signed-off-by: xinyual <xinyual@amazon.com>

remove useless change

651db98

Signed-off-by: xinyual <xinyual@amazon.com>

add time functions

706f9b0

Signed-off-by: xinyual <xinyual@amazon.com>

fix date/utc_data/timestamp

8c0dd3b

Signed-off-by: xinyual <xinyual@amazon.com>

add unix_timestamp(half)

d0f4f38

Signed-off-by: xinyual <xinyual@amazon.com>

continue to fix unixtimestamp

2f29ab7

Signed-off-by: xinyual <xinyual@amazon.com>

fix unixtimestamp

8fc4b39

Signed-off-by: xinyual <xinyual@amazon.com>

add timestamp

6eb9204

Signed-off-by: xinyual <xinyual@amazon.com>

fix timestamp

5b75398

Signed-off-by: xinyual <xinyual@amazon.com>

Implement DATEADD function

61deedf

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Implement DATE_SUB function

60a2807

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

WIP: Implement TIMEADD

85798af

Implement ADDTIME

f4c08cf

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Implement DAY_OF_WEEK and DAY_OF_YEAR

02be4c4

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

LantaoJin reviewed Apr 1, 2025

View reviewed changes

qianheng-aws reviewed Apr 1, 2025

View reviewed changes

xinyual added 7 commits April 1, 2025 19:03

add copy right

720b0e7

Signed-off-by: xinyual <xinyual@amazon.com>

change compare to local date time

a82dc58

Signed-off-by: xinyual <xinyual@amazon.com>

Merge remote-tracking branch 'origin/main' into withUDT

b922857

change name after merge

3469557

Signed-off-by: xinyual <xinyual@amazon.com>

remove useless code

e9165fd

Signed-off-by: xinyual <xinyual@amazon.com>

fix bug

d413bdb

Signed-off-by: xinyual <xinyual@amazon.com>

ignore flaky test

b80ea40

Signed-off-by: xinyual <xinyual@amazon.com>

qianheng-aws reviewed Apr 2, 2025

View reviewed changes

LantaoJin reviewed Apr 2, 2025

View reviewed changes

xinyual and others added 4 commits April 2, 2025 18:09

modify IT message information

8ae7d16

Signed-off-by: xinyual <xinyual@amazon.com>

Fix date / time formatting (always prefer Arabic digits instead of lo…

30941ee

…cal digits) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Merge pull request #23 from yuancu/fix-date-format

a1e20ad

Fix date / time formatting

apply spot

c686197

Signed-off-by: xinyual <xinyual@amazon.com>

LantaoJin previously approved these changes Apr 2, 2025

View reviewed changes

yuancu and others added 2 commits April 3, 2025 10:21

Reuse PlanUtils.intervalUnitToSpanUnit

d510c84

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Merge pull request #24 from yuancu/fix-datetime

9d8e849

Reuse PlanUtils.intervalUnitToSpanUnit

xinyual dismissed LantaoJin’s stale review via 9d8e849 April 3, 2025 02:30

xinyual added 2 commits April 3, 2025 10:31

revert useless code

508fef8

Signed-off-by: xinyual <xinyual@amazon.com>

apply spotless and revert useless change

345beca

Signed-off-by: xinyual <xinyual@amazon.com>

qianheng-aws approved these changes Apr 3, 2025

View reviewed changes

LantaoJin approved these changes Apr 3, 2025

View reviewed changes

LantaoJin merged commit 920f711 into opensearch-project:main Apr 3, 2025
21 checks passed

yuancu mentioned this pull request Apr 3, 2025

Validate arguments for datetime, string, and math UDFs #3512

Closed

8 tasks

noCharger mentioned this pull request Apr 9, 2025

Add release notes on version 3.0.0.0-beta1 #3529

Merged

2 tasks

This was referenced Apr 11, 2025

[BUG] timestampdiff return wrong schema in V2 #3475

Closed

[BUG] percentile returns wrong value in V2 #3495

Closed

LantaoJin mentioned this pull request Aug 15, 2025

[GitHub Request] Add yuancu as maintainer of OpenSearch SQL opensearch-project/.github#374

Closed

		@@ -0,0 +1,25 @@
		package org.opensearch.sql.calcite.udf.datetimeUDF;

		@@ -0,0 +1,37 @@
		package org.opensearch.sql.calcite.udf.datetimeUDF;

Conversation

xinyual commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xinyual Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qianheng-aws Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xinyual Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

xinyual commented Mar 24, 2025 •

edited

Loading

xinyual Apr 1, 2025 •

edited

Loading

qianheng-aws Apr 1, 2025 •

edited

Loading

xinyual Apr 1, 2025 •

edited

Loading