Skip to content

Commit 4630a3e

Browse files
andygroveclaude
andcommitted
docs: add missing expressions to user guide
Add documentation for expressions that were implemented but not documented in the supported expressions list: - Left (string function) - DateDiff, DateFormat, LastDay, UnixDate, UnixTimestamp (date/time) - Sha1 (hashing) - JsonToStructs (struct) Also fixes TruncTimestamp SQL from `trunc_date` to `date_trunc`. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent ea26629 commit 4630a3e

File tree

1 file changed

+15
-7
lines changed

1 file changed

+15
-7
lines changed

docs/source/user-guide/latest/expressions.md

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ Expressions that are not Spark-compatible will fall back to Spark by default and
6868
| Contains | Yes | |
6969
| EndsWith | Yes | |
7070
| InitCap | No | Behavior is different in some cases, such as hyphenated names. |
71+
| Left | Yes | Length argument must be a literal value |
7172
| Length | Yes | |
7273
| Like | Yes | |
7374
| Lower | No | Results can vary depending on locale and character set. Requires `spark.comet.caseConversion.enabled=true` |
@@ -94,15 +95,20 @@ Expressions that are not Spark-compatible will fall back to Spark by default and
9495
| Expression | SQL | Spark-Compatible? | Compatibility Notes |
9596
| -------------- | ---------------------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------- |
9697
| DateAdd | `date_add` | Yes | |
98+
| DateDiff | `datediff` | Yes | |
99+
| DateFormat | `date_format` | Yes | Partial support. Only specific format patterns are supported. |
97100
| DateSub | `date_sub` | Yes | |
98101
| DatePart | `date_part(field, source)` | Yes | Supported values of `field`: `year`/`month`/`week`/`day`/`dayofweek`/`dayofweek_iso`/`doy`/`quarter`/`hour`/`minute` |
99102
| Extract | `extract(field FROM source)` | Yes | Supported values of `field`: `year`/`month`/`week`/`day`/`dayofweek`/`dayofweek_iso`/`doy`/`quarter`/`hour`/`minute` |
100103
| FromUnixTime | `from_unixtime` | No | Does not support format, supports only -8334601211038 <= sec <= 8210266876799 |
101104
| Hour | `hour` | Yes | |
105+
| LastDay | `last_day` | Yes | |
102106
| Minute | `minute` | Yes | |
103107
| Second | `second` | Yes | |
104108
| TruncDate | `trunc` | Yes | |
105-
| TruncTimestamp | `trunc_date` | Yes | |
109+
| TruncTimestamp | `date_trunc` | Yes | |
110+
| UnixDate | `unix_date` | Yes | |
111+
| UnixTimestamp | `unix_timestamp` | Yes | |
106112
| Year | `year` | Yes | |
107113
| Month | `month` | Yes | |
108114
| DayOfMonth | `day`/`dayofmonth` | Yes | |
@@ -163,6 +169,7 @@ Expressions that are not Spark-compatible will fall back to Spark by default and
163169
| ----------- | ----------------- |
164170
| Md5 | Yes |
165171
| Murmur3Hash | Yes |
172+
| Sha1 | Yes |
166173
| Sha2 | Yes |
167174
| XxHash64 | Yes |
168175

@@ -256,12 +263,13 @@ Comet supports using the following aggregate functions within window contexts wi
256263

257264
## Struct Expressions
258265

259-
| Expression | Spark-Compatible? |
260-
| -------------------- | ----------------- |
261-
| CreateNamedStruct | Yes |
262-
| GetArrayStructFields | Yes |
263-
| GetStructField | Yes |
264-
| StructsToJson | Yes |
266+
| Expression | Spark-Compatible? | Compatibility Notes |
267+
| -------------------- | ----------------- | ------------------------------------------ |
268+
| CreateNamedStruct | Yes | |
269+
| GetArrayStructFields | Yes | |
270+
| GetStructField | Yes | |
271+
| JsonToStructs | No | Partial support. Requires explicit schema. |
272+
| StructsToJson | Yes | |
265273

266274
## Conversion Expressions
267275

0 commit comments

Comments
 (0)