Skip to content

Commit 7a5fd4a

Browse files
wangyumgatorsmile
authored andcommitted
[SPARK-18874][SQL][FOLLOW-UP] Improvement type mismatched message
## What changes were proposed in this pull request? Improvement `IN` predicate type mismatched message: ```sql Mismatched columns: [(, t, 4, ., `, t, 4, a, `, :, d, o, u, b, l, e, ,, , t, 5, ., `, t, 5, a, `, :, d, e, c, i, m, a, l, (, 1, 8, ,, 0, ), ), (, t, 4, ., `, t, 4, c, `, :, s, t, r, i, n, g, ,, , t, 5, ., `, t, 5, c, `, :, b, i, g, i, n, t, )] ``` After this patch: ```sql Mismatched columns: [(t4.`t4a`:double, t5.`t5a`:decimal(18,0)), (t4.`t4c`:string, t5.`t5c`:bigint)] ``` ## How was this patch tested? unit tests Author: Yuming Wang <[email protected]> Closes apache#21863 from wangyum/SPARK-18874.
1 parent 78e0a72 commit 7a5fd4a

File tree

3 files changed

+70
-14
lines changed

3 files changed

+70
-14
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,7 @@ case class In(value: Expression, list: Seq[Expression]) extends Predicate {
189189
} else {
190190
val mismatchedColumns = valExprs.zip(childOutputs).flatMap {
191191
case (l, r) if l.dataType != r.dataType =>
192-
s"(${l.sql}:${l.dataType.catalogString}, ${r.sql}:${r.dataType.catalogString})"
192+
Seq(s"(${l.sql}:${l.dataType.catalogString}, ${r.sql}:${r.dataType.catalogString})")
193193
case _ => None
194194
}
195195
TypeCheckResult.TypeCheckFailure(

sql/core/src/test/resources/sql-tests/inputs/subquery/negative-cases/subq-input-typecheck.sql

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,14 @@ CREATE TEMPORARY VIEW t3 AS SELECT * FROM VALUES
1313
(3, 1, 2)
1414
AS t3(t3a, t3b, t3c);
1515

16+
CREATE TEMPORARY VIEW t4 AS SELECT * FROM VALUES
17+
(CAST(1 AS DOUBLE), CAST(2 AS STRING), CAST(3 AS STRING))
18+
AS t1(t4a, t4b, t4c);
19+
20+
CREATE TEMPORARY VIEW t5 AS SELECT * FROM VALUES
21+
(CAST(1 AS DECIMAL(18, 0)), CAST(2 AS STRING), CAST(3 AS BIGINT))
22+
AS t1(t5a, t5b, t5c);
23+
1624
-- TC 01.01
1725
SELECT
1826
( SELECT max(t2b), min(t2b)
@@ -44,4 +52,10 @@ WHERE
4452
(t1a, t1b) IN (SELECT t2a
4553
FROM t2
4654
WHERE t1a = t2a);
47-
55+
-- TC 01.05
56+
SELECT * FROM t4
57+
WHERE
58+
(t4a, t4b, t4c) IN (SELECT t5a,
59+
t5b,
60+
t5c
61+
FROM t5);

sql/core/src/test/resources/sql-tests/results/subquery/negative-cases/subq-input-typecheck.sql.out

Lines changed: 54 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
-- Automatically generated by SQLQueryTestSuite
2-
-- Number of queries: 7
2+
-- Number of queries: 10
33

44

55
-- !query 0
@@ -33,44 +33,64 @@ struct<>
3333

3434

3535
-- !query 3
36+
CREATE TEMPORARY VIEW t4 AS SELECT * FROM VALUES
37+
(CAST(1 AS DOUBLE), CAST(2 AS STRING), CAST(3 AS STRING))
38+
AS t1(t4a, t4b, t4c)
39+
-- !query 3 schema
40+
struct<>
41+
-- !query 3 output
42+
43+
44+
45+
-- !query 4
46+
CREATE TEMPORARY VIEW t5 AS SELECT * FROM VALUES
47+
(CAST(1 AS DECIMAL(18, 0)), CAST(2 AS STRING), CAST(3 AS BIGINT))
48+
AS t1(t5a, t5b, t5c)
49+
-- !query 4 schema
50+
struct<>
51+
-- !query 4 output
52+
53+
54+
55+
-- !query 5
3656
SELECT
3757
( SELECT max(t2b), min(t2b)
3858
FROM t2
3959
WHERE t2.t2b = t1.t1b
4060
GROUP BY t2.t2b
4161
)
4262
FROM t1
43-
-- !query 3 schema
63+
-- !query 5 schema
4464
struct<>
45-
-- !query 3 output
65+
-- !query 5 output
4666
org.apache.spark.sql.AnalysisException
4767
Scalar subquery must return only one column, but got 2;
4868

4969

50-
-- !query 4
70+
-- !query 6
5171
SELECT
5272
( SELECT max(t2b), min(t2b)
5373
FROM t2
5474
WHERE t2.t2b > 0
5575
GROUP BY t2.t2b
5676
)
5777
FROM t1
58-
-- !query 4 schema
78+
-- !query 6 schema
5979
struct<>
60-
-- !query 4 output
80+
-- !query 6 output
6181
org.apache.spark.sql.AnalysisException
6282
Scalar subquery must return only one column, but got 2;
6383

6484

65-
-- !query 5
85+
-- !query 7
6686
SELECT * FROM t1
6787
WHERE
6888
t1a IN (SELECT t2a, t2b
6989
FROM t2
7090
WHERE t1a = t2a)
71-
-- !query 5 schema
91+
-- !query 7 schema
7292
struct<>
73-
-- !query 5 output
93+
-- !query 7 output
7494
org.apache.spark.sql.AnalysisException
7595
cannot resolve '(t1.`t1a` IN (listquery(t1.`t1a`)))' due to data type mismatch:
7696
The number of columns in the left hand side of an IN subquery does not match the
@@ -83,15 +103,15 @@ Right side columns:
83103
[t2.`t2a`, t2.`t2b`].;
84104

85105

86-
-- !query 6
106+
-- !query 8
87107
SELECT * FROM T1
88108
WHERE
89109
(t1a, t1b) IN (SELECT t2a
90110
FROM t2
91111
WHERE t1a = t2a)
92-
-- !query 6 schema
112+
-- !query 8 schema
93113
struct<>
94-
-- !query 6 output
114+
-- !query 8 output
95115
org.apache.spark.sql.AnalysisException
96116
cannot resolve '(named_struct('t1a', t1.`t1a`, 't1b', t1.`t1b`) IN (listquery(t1.`t1a`)))' due to data type mismatch:
97117
The number of columns in the left hand side of an IN subquery does not match the
@@ -102,3 +122,25 @@ Left side columns:
102122
[t1.`t1a`, t1.`t1b`].
103123
Right side columns:
104124
[t2.`t2a`].;
125+
126+
127+
-- !query 9
128+
SELECT * FROM t4
129+
WHERE
130+
(t4a, t4b, t4c) IN (SELECT t5a,
131+
t5b,
132+
t5c
133+
FROM t5)
134+
-- !query 9 schema
135+
struct<>
136+
-- !query 9 output
137+
org.apache.spark.sql.AnalysisException
138+
cannot resolve '(named_struct('t4a', t4.`t4a`, 't4b', t4.`t4b`, 't4c', t4.`t4c`) IN (listquery()))' due to data type mismatch:
139+
The data type of one or more elements in the left hand side of an IN subquery
140+
is not compatible with the data type of the output of the subquery
141+
Mismatched columns:
142+
[(t4.`t4a`:double, t5.`t5a`:decimal(18,0)), (t4.`t4c`:string, t5.`t5c`:bigint)]
143+
Left side:
144+
[double, string, string].
145+
Right side:
146+
[decimal(18,0), string, bigint].;

0 commit comments

Comments
 (0)