Skip to content

Commit fc987a7

Browse files
authored
Update apache-spark-troubleshoot-job-fails-invalidclassexception.md
This is to address [User Story 1621754](https://dev.azure.com/mseng/TechnicalContent/_queries/edit/1621754/?triage=true), which asks for a clarification of "shaded jar".
1 parent a1bc21a commit fc987a7

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

articles/hdinsight/spark/apache-spark-troubleshoot-job-fails-invalidclassexception.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,10 @@ org.apache.commons.lang3.time.FastDateFormat; local class incompatible: stream c
2929

3030
## Cause
3131

32-
This error can be caused by adding an additional jar to the `spark.yarn.jars` config, which is a “shaded” jar that includes a different version of `commons-lang3` package and introduces a class mismatch. By default, Spark 2.1/2/3 uses version 3.5 of `commons-lang3`.
32+
This error can be caused by adding an additional jar to the `spark.yarn.jars` config, specifically a shaded jar that includes a different version of the `commons-lang3` package and introduces a class mismatch. By default, Spark 2.1/2/3 uses version 3.5 of `commons-lang3`.
33+
34+
> [!TIP]
35+
> To shade a library is to put its contents into your own jar, changing its package. This differs from packaging the library, which is putting the library into your own jar without repackaging.
3336
3437
## Resolution
3538

0 commit comments

Comments
 (0)