Skip to content

Commit 1345a53

Browse files
authored
docs: Documentation updates for LocalTableScan and WindowExec (#2742)
1 parent 9b91c25 commit 1345a53

File tree

3 files changed

+25
-2
lines changed

3 files changed

+25
-2
lines changed

docs/source/user-guide/latest/compatibility.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,11 @@ Comet uses the Rust regexp crate for evaluating regular expressions, and this ha
6868
regular expression engine. Comet will fall back to Spark for patterns that are known to produce different results, but
6969
this can be overridden by setting `spark.comet.regexp.allowIncompatible=true`.
7070

71+
## Window Functions
72+
73+
Comet's support for window functions is incomplete and known to be incorrect. It is disabled by default and
74+
should not be used in production. The feature will be enabled in a future release. Tracking issue: [#2721](https://github.com/apache/datafusion-comet/issues/2721).
75+
7176
## Cast
7277

7378
Cast operations in Comet fall into three levels of support:

docs/source/user-guide/latest/expressions.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,23 @@ incompatible expressions.
202202
| VariancePop | | Yes | |
203203
| VarianceSamp | | Yes | |
204204

205+
## Window Functions
206+
207+
```{warning}
208+
Window support is disabled by default due to known correctness issues. Tracking issue: [#2721](https://github.com/apache/datafusion-comet/issues/2721).
209+
```
210+
211+
Comet supports using the following aggregate functions within window contexts with PARTITION BY and ORDER BY clauses.
212+
213+
| Expression | Spark-Compatible? | Compatibility Notes |
214+
| ---------- | ----------------- | -------------------------------------------- |
215+
| Count | Yes | |
216+
| Max | Yes | |
217+
| Min | Yes | |
218+
| Sum | Yes | |
219+
220+
**Note:** Dedicated window functions such as `rank`, `dense_rank`, `row_number`, `lag`, `lead`, `ntile`, `cume_dist`, `percent_rank`, and `nth_value` are not currently supported and will fall back to Spark.
221+
205222
## Array Expressions
206223

207224
| Expression | Spark-Compatible? | Compatibility Notes |

docs/source/user-guide/latest/operators.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ The following Spark operators are currently replaced with native versions. Query
2323
not supported by Comet will fall back to regular Spark execution.
2424

2525
| Operator | Spark-Compatible? | Compatibility Notes |
26-
| ----------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------ |
26+
|-------------------------|-------------------|--------------------------------------------------------------------------------------------------------------------|
2727
| BatchScanExec | Yes | Supports Parquet files and Apache Iceberg Parquet scans. See the [Comet Compatibility Guide] for more information. |
2828
| BroadcastExchangeExec | Yes | |
2929
| BroadcastHashJoinExec | Yes | |
@@ -33,13 +33,14 @@ not supported by Comet will fall back to regular Spark execution.
3333
| GlobalLimitExec | Yes | |
3434
| HashAggregateExec | Yes | |
3535
| LocalLimitExec | Yes | |
36+
| LocalTableScanExec | No | Experimental and disabled by default. |
3637
| ObjectHashAggregateExec | Yes | Supports a limited number of aggregates, such as `bloom_filter_agg`. |
3738
| ProjectExec | Yes | |
3839
| ShuffleExchangeExec | Yes | |
3940
| ShuffledHashJoinExec | Yes | |
4041
| SortExec | Yes | |
4142
| SortMergeJoinExec | Yes | |
4243
| UnionExec | Yes | |
43-
| WindowExec | Yes | |
44+
| WindowExec | No | Disabled by default due to known correctness issues. |
4445

4546
[Comet Compatibility Guide]: compatibility.md

0 commit comments

Comments
 (0)