Skip to content

Commit 5d38830

Browse files
committed
[SPARK-53781] Exclude Spark's transitive dependencies consistently across modules
### What changes were proposed in this pull request? This PR aims to exclude `Spark`'s transitive dependencies consistently across modules. ### Why are the changes needed? Currently `spark-operator` and `spark-submission-worker` modules have heterogeneous exclusion rules against Apache Spark. To be complete, we need to be consistent. https://github.com/apache/spark-kubernetes-operator/blob/2366928e114e663d77ef0c890b59fab72a26ebc1/spark-operator/build.gradle#L41-L51 https://github.com/apache/spark-kubernetes-operator/blob/2366928e114e663d77ef0c890b59fab72a26ebc1/spark-submission-worker/build.gradle#L23-L33 This PR proposes **the following sorted list** which is a union of the existing exclusions for both modules. ``` exclude group: "com.github.luben" exclude group: "commons-collections", module: "commons-collections' exclude group: "io.fabric8" exclude group: "io.netty", module: "netty-tcnative-boringssl-static" exclude group: "io.netty", module: "netty-tcnative-classes" exclude group: "org.apache.logging.log4j" exclude group: "org.fusesource.leveldbjni" exclude group: "org.lz4" exclude group: "org.rocksdb" exclude group: "org.slf4j" exclude group: "org.xerial.snappy" ``` ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #361 from dongjoon-hyun/SPARK-53781. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 432f64a commit 5d38830

File tree

2 files changed

+9
-2
lines changed

2 files changed

+9
-2
lines changed

spark-operator/build.gradle

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ dependencies {
4040
implementation(libs.metrics.jvm)
4141
compileOnly(libs.spark.core) {
4242
exclude group: "com.github.luben"
43+
exclude group: "commons-collections", module: "commons-collections"
4344
exclude group: "io.netty", module: "netty-tcnative-boringssl-static"
4445
exclude group: "io.netty", module: "netty-tcnative-classes"
4546
exclude group: "org.apache.logging.log4j"

spark-submission-worker/build.gradle

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,17 +19,23 @@
1919
dependencies {
2020
implementation project(":spark-operator-api")
2121
implementation(libs.kubernetes.client)
22+
implementation(libs.log4j.api)
23+
implementation(libs.log4j.core)
24+
implementation(libs.log4j.slf4j2.impl)
25+
implementation(libs.slf4j.api)
2226

2327
implementation(libs.spark.kubernetes) {
24-
exclude group: "io.fabric8"
2528
exclude group: "com.github.luben"
29+
exclude group: "commons-collections", module: "commons-collections"
30+
exclude group: "io.fabric8"
2631
exclude group: "io.netty", module: "netty-tcnative-boringssl-static"
2732
exclude group: "io.netty", module: "netty-tcnative-classes"
33+
exclude group: "org.apache.logging.log4j"
2834
exclude group: "org.fusesource.leveldbjni"
2935
exclude group: "org.lz4"
3036
exclude group: "org.rocksdb"
37+
exclude group: "org.slf4j"
3138
exclude group: "org.xerial.snappy"
32-
exclude group: 'commons-collections', module: 'commons-collections'
3339
}
3440

3541
compileOnly(libs.lombok)

0 commit comments

Comments
 (0)