You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-25700][SQL] Creates ReadSupport in only Append Mode in Data Source V2 write path
## What changes were proposed in this pull request?
This PR proposes to avoid to make a readsupport and read schema when it writes in other save modes.
apache@5fef6e3 happened to create a readsupport in write path, which ended up with reading schema from readsupport at write path.
This breaks `spark.range(1).format("source").write.save("non-existent-path")` case since there's no way to read the schema from "non-existent-path".
See also apache#22009 (comment)
See also apache#22697
See also http://apache-spark-developers-list.1001551.n3.nabble.com/Possible-bug-in-DatasourceV2-td25343.html
## How was this patch tested?
Unit test and manual tests.
Closesapache#22688 from HyukjinKwon/append-revert-2.
Authored-by: hyukjinkwon <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
0 commit comments