docs: Various documentation updates #2674
Conversation
| ## Array Expressions | ||
| Expressions that are not 100% Spark-compatible will fall back to Spark by default and can be enabled by setting | ||
| `spark.comet.expression.EXPRNAME.allowIncompatible=true`, where `EXPRNAME` is the Spark expression class name. See | ||
| the [Comet Supported Expressions Guide](expressions.md) for more information on this configuration setting. | ||
|
|
||
| Comet has experimental support for a number of array expressions. These are experimental and currently marked | ||
| as incompatible and can be enabled by setting `spark.comet.expression.allowIncompatible=true`. |
There was a problem hiding this comment.
The previous section explains incompatible expressions. There is no need to say anything special about array expressions.
| There is a known bug with using count(distinct) within aggregate queries, where each NaN value will be counted | ||
| separately [#1824](https://github.com/apache/datafusion-comet/issues/1824). |
comphead
left a comment
comphead
left a comment
There was a problem hiding this comment.
Thanks @andygrove
I would also include @parthchandra flow-class-landscape diagram how everything is interconnected https://docs.google.com/drawings/d/1yYyyGVlItfJ5G1m-3eYJYeg6p4Jce3TRxY83MQOrV74
Thanks @comphead. I will create a follow-on PR to add a text version of Parth's diagram. I have an AI generated version ready to submit. |
|
2 participants
Which issue does this PR close?
N/A
Rationale for this change
The main motivation was to move the very detailed description of the different Parquet scan implementations out of the user guide and into a dedicated page in the contributor guide. Users should not need to know these details.
I also made some other minor updates.
What changes are included in this PR?
How are these changes tested?