Skip to content

Commit 7ebd395

Browse files
dongjoon-hyunyaooqinn
authored andcommitted
[SPARK-50729][SQL][TESTS] Remove DebugFilesystem from TPCDS(Collation)?QueryTestSuite
### What changes were proposed in this pull request? This PR aims to remove `DebugFilesystem` from `TPCDS(Collation)?QueryTestSuite` while keeping it in all other places. ### Why are the changes needed? `TPCDS(Collation)?QueryTestSuite` extends `TPCDSBase` and `TPCHBase` respectively and they are based on `TPCBase` which extends `SharedSparkSession`. https://github.com/apache/spark/blob/91f3fdd25852b43095dd5273358fc394ffd11b66/sql/core/src/test/scala/org/apache/spark/sql/TPCBase.scala#L24 And, `SharedSparkSession` always uses `DebugFilesystem`. https://github.com/apache/spark/blob/91f3fdd25852b43095dd5273358fc394ffd11b66/sql/core/src/test/scala/org/apache/spark/sql/test/SharedSparkSession.scala#L68 This PR aims to remove this resetting. ### Does this PR introduce _any_ user-facing change? No, this is a test-only change. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #49359 from dongjoon-hyun/SPARK-50729. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Kent Yao <yao@apache.org>
1 parent aed5d11 commit 7ebd395

File tree

2 files changed

+2
-0
lines changed

2 files changed

+2
-0
lines changed

sql/core/src/test/scala/org/apache/spark/sql/TPCDSCollationQueryTestSuite.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ class TPCDSCollationQueryTestSuite extends QueryTest with TPCDSBase with SQLQuer
6767
// To make output results deterministic
6868
override protected def sparkConf: SparkConf = super.sparkConf
6969
.set(SQLConf.SHUFFLE_PARTITIONS.key, "1")
70+
.remove("spark.hadoop.fs.file.impl")
7071

7172
protected override def createSparkSession: TestSparkSession = {
7273
new TestSparkSession(new SparkContext("local[1]", this.getClass.getSimpleName, sparkConf))

sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ class TPCDSQueryTestSuite extends QueryTest with TPCDSBase with SQLQueryTestHelp
6262
// To make output results deterministic
6363
override protected def sparkConf: SparkConf = super.sparkConf
6464
.set(SQLConf.SHUFFLE_PARTITIONS.key, "1")
65+
.remove("spark.hadoop.fs.file.impl")
6566

6667
protected override def createSparkSession: TestSparkSession = {
6768
new TestSparkSession(new SparkContext("local[1]", this.getClass.getSimpleName, sparkConf))

0 commit comments

Comments
 (0)