Skip to content

Commit a5fb5b6

Browse files
忍冬srowen
authored andcommitted
[SPARK-25235][BUILD][SHELL][FOLLOWUP] Fix repl compile for 2.12
## What changes were proposed in this pull request? Error messages from https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/183/ ``` [INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) spark-repl_2.12 --- [INFO] Using zinc server for incremental compilation [warn] Pruning sources from previous analysis, due to incompatible CompileSetup. [info] Compiling 6 Scala sources to /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/target/scala-2.12/classes... [error] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala:80: overriding lazy value importableSymbolsWithRenames in class ImportHandler of type List[(this.intp.global.Symbol, this.intp.global.Name)]; [error] lazy value importableSymbolsWithRenames needs `override' modifier [error] lazy val importableSymbolsWithRenames: List[(Symbol, Name)] = { [error] ^ [warn] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala:53: variable addedClasspath in class ILoop is deprecated (since 2.11.0): use reset, replay or require to update class path [warn] if (addedClasspath != "") { [warn] ^ [warn] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala:54: variable addedClasspath in class ILoop is deprecated (since 2.11.0): use reset, replay or require to update class path [warn] settings.classpath append addedClasspath [warn] ^ [warn] two warnings found [error] one error found [error] Compile failed at Aug 29, 2018 5:28:22 PM [0.679s] ``` Readd the profile for `scala-2.12`. Using `-Pscala-2.12` will overrides `extra.source.dir` and `extra.testsource.dir` with two non-exist directories. ## How was this patch tested? First, make sure it compiles. ``` dev/change-scala-version.sh 2.12 mvn -Pscala-2.12 -DskipTests compile install ``` Then, make a distribution to try the repl: `./dev/make-distribution.sh --name custom-spark --tgz -Phadoop-2.7 -Phive -Pyarn -Pscala-2.12` ``` 18/08/30 16:04:50 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). Spark context Web UI available at http://172.16.131.140:4040 Spark context available as 'sc' (master = local[*], app id = local-1535616298812). Spark session available as 'spark'. Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 2.4.0-SNAPSHOT /_/ Using Scala version 2.12.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112) Type in expressions to have them evaluated. Type :help for more information. scala> spark.sql("select percentile(key, 1) from values (1, 1),(2, 1) T(key, value)").show +-------------------------------------+ |percentile(key, CAST(1 AS DOUBLE), 1)| +-------------------------------------+ | 2.0| +-------------------------------------+ ``` Closes apache#22280 from sadhen/SPARK_24785_FOLLOWUP. Authored-by: 忍冬 <[email protected]> Signed-off-by: Sean Owen <[email protected]>
1 parent c685b5f commit a5fb5b6

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

repl/pom.xml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,4 +167,14 @@
167167
</plugins>
168168
</build>
169169

170+
<profiles>
171+
<profile>
172+
<id>scala-2.12</id>
173+
<properties>
174+
<extra.source.dir>scala-2.12/src/main/scala</extra.source.dir>
175+
<extra.testsource.dir>scala-2.12/src/test/scala</extra.testsource.dir>
176+
</properties>
177+
</profile>
178+
</profiles>
179+
170180
</project>

0 commit comments

Comments
 (0)