Skip to content

Commit 278984d

Browse files
dongjoon-hyunyhuai
authored andcommitted
[SPARK-25019][BUILD] Fix orc dependency to use the same exclusion rules
## What changes were proposed in this pull request? During upgrading Apache ORC to 1.5.2 ([SPARK-24576](https://issues.apache.org/jira/browse/SPARK-24576)), `sql/core` module overrides the exclusion rules of parent pom file and it causes published `spark-sql_2.1X` artifacts have incomplete exclusion rules ([SPARK-25019](https://issues.apache.org/jira/browse/SPARK-25019)). This PR fixes it by moving the newly added exclusion rule to the parent pom. This also fixes the sbt build hack introduced at that time. ## How was this patch tested? Pass the existing dependency check and the tests. Author: Dongjoon Hyun <[email protected]> Closes apache#22003 from dongjoon-hyun/SPARK-25019.
1 parent 51e2b38 commit 278984d

File tree

2 files changed

+4
-28
lines changed

2 files changed

+4
-28
lines changed

pom.xml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1743,6 +1743,10 @@
17431743
<groupId>org.apache.hadoop</groupId>
17441744
<artifactId>hadoop-common</artifactId>
17451745
</exclusion>
1746+
<exclusion>
1747+
<groupId>org.apache.hadoop</groupId>
1748+
<artifactId>hadoop-hdfs</artifactId>
1749+
</exclusion>
17461750
<exclusion>
17471751
<groupId>org.apache.hive</groupId>
17481752
<artifactId>hive-storage-api</artifactId>

sql/core/pom.xml

Lines changed: 0 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -90,39 +90,11 @@
9090
<groupId>org.apache.orc</groupId>
9191
<artifactId>orc-core</artifactId>
9292
<classifier>${orc.classifier}</classifier>
93-
<exclusions>
94-
<exclusion>
95-
<groupId>org.apache.hadoop</groupId>
96-
<artifactId>hadoop-hdfs</artifactId>
97-
</exclusion>
98-
<!--
99-
orc-core:nohive doesn't have this dependency, but we adds this to prevent
100-
sbt from getting confused.
101-
-->
102-
<exclusion>
103-
<groupId>org.apache.hive</groupId>
104-
<artifactId>hive-storage-api</artifactId>
105-
</exclusion>
106-
</exclusions>
10793
</dependency>
10894
<dependency>
10995
<groupId>org.apache.orc</groupId>
11096
<artifactId>orc-mapreduce</artifactId>
11197
<classifier>${orc.classifier}</classifier>
112-
<exclusions>
113-
<exclusion>
114-
<groupId>org.apache.hadoop</groupId>
115-
<artifactId>hadoop-hdfs</artifactId>
116-
</exclusion>
117-
<!--
118-
orc-core:nohive doesn't have this dependency, but we adds this to prevent
119-
sbt from getting confused.
120-
-->
121-
<exclusion>
122-
<groupId>org.apache.hive</groupId>
123-
<artifactId>hive-storage-api</artifactId>
124-
</exclusion>
125-
</exclusions>
12698
</dependency>
12799
<dependency>
128100
<groupId>org.apache.parquet</groupId>

0 commit comments

Comments
 (0)