Commit 5d8249f
authored
fix: Fix and Refactor Spark
## Which issue does this PR close?
- Closes #20483.
## Rationale for this change
Currently, Spark `shuffle` function returns following error message when
`seed` is `null`. This needs to be fixed by exposing `NULL` instead of
`'Int64'`.
**Current:**
```
query error
SELECT shuffle([2, 1], NULL);
----
DataFusion error: Execution error: shuffle seed must be Int64 type, got 'Int64'
```
**New:**
```
query error DataFusion error: Execution error: shuffle seed must be Int64 type but got 'NULL'
SELECT shuffle([1, 2, 3], NULL);
```
In addition to this fix, this PR also introduces following refactoring
to `shuffle` function:
- Combining args validation checks with `single` error message,
- Extending current error message with expected data types:
```
Current:
shuffle does not support type '{array_type}'.
New:
shuffle does not support type '{array_type}'; expected types: List, LargeList, FixedSizeList or Null."
```
- Adding new UT coverages for both `shuffle.rs` and `shuffle.slt`.
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
Yes, being added new UT cases.
## Are there any user-facing changes?
Yes, updating Spark `shuffle` functions error messages.shuffle function (#20484)1 parent e567cb9 commit 5d8249f
File tree
2 files changed
+18
-8
lines changed- datafusion
- spark/src/function/array
- sqllogictest/test_files/spark/array
2 files changed
+18
-8
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
| 108 | + | |
| 109 | + | |
113 | 110 | | |
114 | 111 | | |
115 | 112 | | |
| |||
131 | 128 | | |
132 | 129 | | |
133 | 130 | | |
134 | | - | |
| 131 | + | |
135 | 132 | | |
136 | 133 | | |
137 | | - | |
| 134 | + | |
138 | 135 | | |
139 | 136 | | |
140 | 137 | | |
| |||
164 | 161 | | |
165 | 162 | | |
166 | 163 | | |
167 | | - | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
168 | 168 | | |
169 | 169 | | |
170 | 170 | | |
| |||
Lines changed: 10 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
107 | 107 | | |
108 | 108 | | |
109 | 109 | | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
110 | 120 | | |
111 | 121 | | |
112 | 122 | | |
| |||
0 commit comments