Skip to content

Commit afecdc2

Browse files
committed
[SPARK-52341] Upgrade Spark to 3.5.6 from 3.5.5 in Spark 3 and Iceberg integration tests
### What changes were proposed in this pull request? This PR aims to upgrade Spark to 3.5.6 from 3.5.5 in Spark 3 and Iceberg integration tests ### Why are the changes needed? Since Apache Spark 3.5.6 is the latest release, 3.5.5 is deleted. It causes CI failures currently. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #183 from dongjoon-hyun/SPARK-52341. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 422dd14 commit afecdc2

File tree

1 file changed

+8
-8
lines changed

1 file changed

+8
-8
lines changed

.github/workflows/build_and_test.yml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -128,11 +128,11 @@ jobs:
128128
java-version: 17
129129
- name: Test
130130
run: |
131-
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.5/spark-3.5.5-bin-hadoop3.tgz?action=download
132-
tar xvfz spark-3.5.5-bin-hadoop3.tgz
133-
mv spark-3.5.5-bin-hadoop3 /tmp/spark
131+
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.6/spark-3.5.6-bin-hadoop3.tgz?action=download
132+
tar xvfz spark-3.5.6-bin-hadoop3.tgz
133+
mv spark-3.5.6-bin-hadoop3 /tmp/spark
134134
cd /tmp/spark/sbin
135-
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.5
135+
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.6
136136
cd -
137137
swift test --no-parallel -c release
138138
@@ -152,11 +152,11 @@ jobs:
152152
java-version: 17
153153
- name: Test
154154
run: |
155-
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.5/spark-3.5.5-bin-hadoop3.tgz?action=download
156-
tar xvfz spark-3.5.5-bin-hadoop3.tgz
157-
mv spark-3.5.5-bin-hadoop3 /tmp/spark
155+
curl -LO https://www.apache.org/dyn/closer.lua/spark/spark-3.5.6/spark-3.5.6-bin-hadoop3.tgz?action=download
156+
tar xvfz spark-3.5.6-bin-hadoop3.tgz
157+
mv spark-3.5.6-bin-hadoop3 /tmp/spark
158158
cd /tmp/spark/sbin
159-
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.5,org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0 -c spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog -c spark.sql.catalog.local.type=hadoop -c spark.sql.catalog.local.warehouse=/tmp/spark/warehouse -c spark.sql.defaultCatalog=local
159+
./start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.6,org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0 -c spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog -c spark.sql.catalog.local.type=hadoop -c spark.sql.catalog.local.warehouse=/tmp/spark/warehouse -c spark.sql.defaultCatalog=local
160160
cd -
161161
swift test --filter DataFrameWriterV2Tests -c release
162162
swift test --filter IcebergTest -c release

0 commit comments

Comments
 (0)