Skip to content

Commit b985c47

Browse files
committed
[SPARK-53683] Use Spark 3.5.7 for Spark 3 integration tests
### What changes were proposed in this pull request? This PR aims to use `Spark 3.5.7` for Spark 3 integration tests. ### Why are the changes needed? Since Apache Spark 3.5.7 is available, we had better use this stable version than 3.5.6. - https://github.com/apache/spark/releases/tag/v3.5.7 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #240 from dongjoon-hyun/SPARK-53683. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 4bbaa9d commit b985c47

File tree

1 file changed

+8
-8
lines changed

1 file changed

+8
-8
lines changed

.github/workflows/build_and_test.yml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -217,11 +217,11 @@ jobs:
217217
run: swift test --filter NOTHING -c release
218218
- name: Test
219219
run: |
220-
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.6/spark-3.5.6-bin-hadoop3.tgz?action=download
221-
tar xvfz spark-3.5.6-bin-hadoop3.tgz && rm spark-3.5.6-bin-hadoop3.tgz
222-
mv spark-3.5.6-bin-hadoop3 /tmp/spark
220+
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.7/spark-3.5.7-bin-hadoop3.tgz?action=download
221+
tar xvfz spark-3.5.7-bin-hadoop3.tgz && rm spark-3.5.7-bin-hadoop3.tgz
222+
mv spark-3.5.7-bin-hadoop3 /tmp/spark
223223
cd /tmp/spark/sbin
224-
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.6
224+
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.7
225225
cd -
226226
swift test --no-parallel -c release
227227
@@ -245,11 +245,11 @@ jobs:
245245
run: swift test --filter NOTHING -c release
246246
- name: Test
247247
run: |
248-
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.6/spark-3.5.6-bin-hadoop3.tgz?action=download
249-
tar xvfz spark-3.5.6-bin-hadoop3.tgz && rm spark-3.5.6-bin-hadoop3.tgz
250-
mv spark-3.5.6-bin-hadoop3 /tmp/spark
248+
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.7/spark-3.5.7-bin-hadoop3.tgz?action=download
249+
tar xvfz spark-3.5.7-bin-hadoop3.tgz && rm spark-3.5.7-bin-hadoop3.tgz
250+
mv spark-3.5.7-bin-hadoop3 /tmp/spark
251251
cd /tmp/spark/sbin
252-
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.6,org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.10.0 -c spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog -c spark.sql.catalog.local.type=hadoop -c spark.sql.catalog.local.warehouse=/tmp/spark/warehouse -c spark.sql.defaultCatalog=local
252+
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.7,org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.10.0 -c spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog -c spark.sql.catalog.local.type=hadoop -c spark.sql.catalog.local.warehouse=/tmp/spark/warehouse -c spark.sql.defaultCatalog=local
253253
cd -
254254
swift test --filter DataFrameWriterV2Tests -c release
255255
swift test --filter IcebergTest -c release

0 commit comments

Comments
 (0)