Commit fc29c91
[SPARK-35561][SQL] Remove leading zeros from empty static number type partition
### What changes were proposed in this pull request?
This PR removes leading zeros from static number type partition when we insert into a partition table with empty partitions.
create table
CREATE TABLE `table_int` ( `id` INT, `c_string` STRING, `p_int` int)
USING parquet PARTITIONED BY (p_int);
insert
insert overwrite table table_int partition (p_int='00011')
select 1, 'c string'
where true ;
|partition|
|---------|
|p_int=11|
insert overwrite table table_int partition (p_int='00012')
select 1, 'c string'
where false ;
|partition|
|---------|
|p_int=00012|
### Why are the changes needed?
This PR creates consistent result when insert empty or non-empty partition
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Add Unit test
Closes apache#33291 from dgd-contributor/35561_insert_integer_partition_fail_when_empty.
Authored-by: dgd-contributor <dgd_contributor@viettel.com.vn>
Signed-off-by: Sean Owen <srowen@gmail.com>1 parent f61d599 commit fc29c91
File tree
2 files changed
+19
-3
lines changed- sql/core/src
- main/scala/org/apache/spark/sql/execution/datasources
- test/scala/org/apache/spark/sql/execution/datasources/parquet
2 files changed
+19
-3
lines changedLines changed: 12 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
352 | 352 | | |
353 | 353 | | |
354 | 354 | | |
355 | | - | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
356 | 358 | | |
357 | 359 | | |
358 | 360 | | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
359 | 368 | | |
360 | 369 | | |
361 | 370 | | |
| |||
523 | 532 | | |
524 | 533 | | |
525 | 534 | | |
526 | | - | |
| 535 | + | |
527 | 536 | | |
528 | | - | |
| 537 | + | |
529 | 538 | | |
530 | 539 | | |
531 | 540 | | |
| |||
Lines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1250 | 1250 | | |
1251 | 1251 | | |
1252 | 1252 | | |
| 1253 | + | |
| 1254 | + | |
| 1255 | + | |
| 1256 | + | |
| 1257 | + | |
| 1258 | + | |
| 1259 | + | |
1253 | 1260 | | |
1254 | 1261 | | |
1255 | 1262 | | |
| |||
0 commit comments