Skip to content

Commit df05fb6

Browse files
jongyouljerryshao
authored andcommitted
[SPARK-23743][SQL] Changed a comparison logic from containing 'slf4j' to starting with 'org.slf4j'
## What changes were proposed in this pull request? isSharedClass returns if some classes can/should be shared or not. It checks if the classes names have some keywords or start with some names. Following the logic, it can occur unintended behaviors when a custom package has `slf4j` inside the package or class name. As I guess, the first intention seems to figure out the class containing `org.slf4j`. It would be better to change the comparison logic to `name.startsWith("org.slf4j")` ## How was this patch tested? This patch should pass all of the current tests and keep all of the current behaviors. In my case, I'm using ProtobufDeserializer to get a table schema from hive tables. Thus some Protobuf packages and names have `slf4j` inside. Without this patch, it cannot be resolved because of ClassCastException from different classloaders. Author: Jongyoul Lee <[email protected]> Closes apache#20860 from jongyoul/SPARK-23743.
1 parent b348901 commit df05fb6

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -179,8 +179,9 @@ private[hive] class IsolatedClientLoader(
179179
val isHadoopClass =
180180
name.startsWith("org.apache.hadoop.") && !name.startsWith("org.apache.hadoop.hive.")
181181

182-
name.contains("slf4j") ||
183-
name.contains("log4j") ||
182+
name.startsWith("org.slf4j") ||
183+
name.startsWith("org.apache.log4j") || // log4j1.x
184+
name.startsWith("org.apache.logging.log4j") || // log4j2
184185
name.startsWith("org.apache.spark.") ||
185186
(sharesHadoopClasses && isHadoopClass) ||
186187
name.startsWith("scala.") ||

0 commit comments

Comments
 (0)