-
Notifications
You must be signed in to change notification settings - Fork 249
Open
Labels
enhancementNew feature or requestNew feature or request
Description
What is the problem the feature request solves?
Comet supports basic operation with windows. The epic tracks progress to increase coverage for windows native execution and mostly driven by #2711 results.
- window: simple COUNT(*) without frame (256 milliseconds)
- window: simple SUM with PARTITION BY (389 milliseconds)
- window: AVG with PARTITION BY and ORDER BY *** FAILED *** (364 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: MIN and MAX with ORDER BY (383 milliseconds)
- window: COUNT with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW *** FAILED *** (196 milliseconds)
Results do not match for query:
--
- window: SUM with ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING *** FAILED *** (190 milliseconds)
Results do not match for query:
--
- window: AVG with ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING *** FAILED *** (183 milliseconds)
Results do not match for query:
--
- window: SUM with ROWS BETWEEN 2 PRECEDING AND CURRENT ROW *** FAILED *** (190 milliseconds)
Results do not match for query:
--
- window: COUNT with ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING *** FAILED *** (183 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: MAX with ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING *** FAILED *** (190 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: ROW_NUMBER with PARTITION BY and ORDER BY *** FAILED *** (186 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: RANK with PARTITION BY and ORDER BY *** FAILED *** (191 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: DENSE_RANK with PARTITION BY and ORDER BY *** FAILED *** (177 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: PERCENT_RANK with PARTITION BY and ORDER BY *** FAILED *** (183 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: NTILE with PARTITION BY and ORDER BY *** FAILED *** (192 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: LAG with default offset *** FAILED *** (192 milliseconds)
Results do not match for query:
--
- window: LAG with offset 2 and default value *** FAILED *** (184 milliseconds)
Results do not match for query:
--
- window: LEAD with default offset *** FAILED *** (176 milliseconds)
Results do not match for query:
--
- window: LEAD with offset 2 and default value *** FAILED *** (173 milliseconds)
Results do not match for query:
--
- window: FIRST_VALUE with default ignore nulls *** FAILED *** (13 milliseconds)
org.apache.spark.SparkUnsupportedOperationException: [ENCODER_NOT_FOUND] Not found an encoder of the type Any to Spark SQL internal representation. Consider to change the input type to one of supported at 'https://spark.apache.org/docs/latest/sql-ref-datatypes.html'.
--
- window: LAST_VALUE with ROWS frame *** FAILED *** (6 milliseconds)
org.apache.spark.SparkUnsupportedOperationException: [ENCODER_NOT_FOUND] Not found an encoder of the type Any to Spark SQL internal representation. Consider to change the input type to one of supported at 'https://spark.apache.org/docs/latest/sql-ref-datatypes.html'.
--
- window: NTH_VALUE with position 2 *** FAILED *** (179 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: CUME_DIST with PARTITION BY and ORDER BY *** FAILED *** (182 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: multiple window functions in single query *** FAILED *** (195 milliseconds)
Results do not match for query:
--
- window: different window specifications in single query *** FAILED *** (225 milliseconds)
Expected only Comet native operators, but found Project.
--
- window: ORDER BY DESC with aggregation *** FAILED *** (187 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: multiple PARTITION BY columns *** FAILED *** (203 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: multiple ORDER BY columns *** FAILED *** (197 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: RANGE BETWEEN with numeric ORDER BY *** FAILED *** (188 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW *** FAILED *** (182 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: complex expression in window function *** FAILED *** (201 milliseconds)
Expected only Comet native operators, but found Project.
--
- window: window function with WHERE clause *** FAILED *** (208 milliseconds)
Expected only Comet native operators, but found Window.
--
- window: window function with GROUP BY *** FAILED *** (273 milliseconds)
Expected only Comet native operators, but found Project.
--
- window: ROWS BETWEEN with negative offset *** FAILED *** (184 milliseconds)
Results do not match for query:
--
- window: all ranking functions together *** FAILED *** (197 milliseconds)
Results do not match for query:
- chore: Expand test coverage for
CometWindowsExec#2711 - External error: Arrow error: Invalid argument error: must either specify a row count or at least one column #2713
- feat: CometWindowExec should correctly account output attributes #2710
- Add support for Window expressions #2705
Describe the potential solution
No response
Additional context
No response
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request