Skip to content
This repository was archived by the owner on Jan 9, 2020. It is now read-only.

Commit 3775dd3

Browse files
xuanyuankinggatorsmile
authored andcommitted
[SPARK-22753][SQL] Get rid of dataSource.writeAndRead
## What changes were proposed in this pull request? As the discussion in apache#16481 and apache#18975 (comment) Currently the BaseRelation returned by `dataSource.writeAndRead` only used in `CreateDataSourceTableAsSelect`, planForWriting and writeAndRead has some common code paths. In this patch I removed the writeAndRead function and added the getRelation function which only use in `CreateDataSourceTableAsSelectCommand` while saving data to non-existing table. ## How was this patch tested? Existing UT Author: Yuanjian Li <[email protected]> Closes apache#19941 from xuanyuanking/SPARK-22753.
1 parent 3fea5c4 commit 3775dd3

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,7 @@ case class InsertIntoDataSourceDirCommand(
6767

6868
val saveMode = if (overwrite) SaveMode.Overwrite else SaveMode.ErrorIfExists
6969
try {
70-
sparkSession.sessionState.executePlan(dataSource.planForWriting(saveMode, query))
71-
dataSource.writeAndRead(saveMode, query)
70+
sparkSession.sessionState.executePlan(dataSource.planForWriting(saveMode, query)).toRdd
7271
} catch {
7372
case ex: AnalysisException =>
7473
logError(s"Failed to write to directory " + storage.locationUri.toString, ex)

0 commit comments

Comments
 (0)