Skip to content

Commit f2e9e1c

Browse files
committed
[SPARK-53380] Add SPARK_LOCAL_IP env variable to MacOS-based integration test jobs
### What changes were proposed in this pull request? This PR aims to recover MacOS-based integration tests by adding `SPARK_LOCAL_IP` environment variables like Apache Spark main repository. ### Why are the changes needed? To recover MacOS-based CI failures. Currently, all MacOS-based integration test GitHub Action jobs are broken. The root cause is that `SparkContext` fails due to `Invalid Spark URL` error like the following because `getHost` is `null` in Java. ``` $ jshell jshell> new java.net.URI("spark://HeartbeatReceiversat12-bq147_186fa0a6-15fa-47f9-ad18-71676ed8dcea-4A992716D7F1.local:49174").getHost() $1 ==> null ``` ``` 25/08/26 06:20:49 INFO SparkConnectServer: Starting Spark session. 25/08/26 06:20:50 INFO SparkContext: Running Spark version 4.1.0-preview1 25/08/26 06:20:50 INFO SparkContext: OS info Mac OS X, 15.5, aarch64 25/08/26 06:20:50 INFO SparkContext: Java version 21.0.8 ... 25/08/26 06:20:50 INFO JettyUtils: Start Jetty 0.0.0.0:4040 for SparkUI 25/08/26 06:20:50 INFO Utils: Successfully started service 'SparkUI' on port 4040. ... 25/08/26 06:20:50 INFO Executor: Starting executor ID driver on host sat12-bq147_186fa0a6-15fa-47f9-ad18-71676ed8dcea-4A992716D7F1.local ... 25/08/26 06:20:50 ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Invalid Spark URL: spark://HeartbeatReceiversat12-bq147_186fa0a6-15fa-47f9-ad18-71676ed8dcea-4A992716D7F1.local:49174 ... 25/08/26 06:20:50 INFO SparkContext: SparkContext is stopping with exitCode 0 from main at NativeMethodAccessorImpl.java:0. ``` - https://github.com/apache/spark-connect-swift/actions/runs/17229744776/job/48881088121 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #230 from dongjoon-hyun/SPARK_LOCAL_IP. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent cd2ed4f commit f2e9e1c

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

.github/workflows/build_and_test.yml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,8 @@ jobs:
115115
integration-test-mac-spark41:
116116
runs-on: macos-15
117117
timeout-minutes: 20
118+
env:
119+
SPARK_LOCAL_IP: localhost
118120
steps:
119121
- uses: actions/checkout@v4
120122
- uses: swift-actions/setup-swift@d10500c1ac8822132eebbd74c48c3372c71d7ff5
@@ -135,6 +137,8 @@ jobs:
135137
integration-test-mac:
136138
runs-on: macos-15
137139
timeout-minutes: 20
140+
env:
141+
SPARK_LOCAL_IP: localhost
138142
steps:
139143
- uses: actions/checkout@v4
140144
- uses: swift-actions/setup-swift@d10500c1ac8822132eebbd74c48c3372c71d7ff5
@@ -156,6 +160,7 @@ jobs:
156160
runs-on: macos-15
157161
timeout-minutes: 20
158162
env:
163+
SPARK_LOCAL_IP: localhost
159164
SPARK_CONNECT_AUTHENTICATE_TOKEN: ${{ github.run_id }}-${{ github.run_attempt }}
160165
steps:
161166
- uses: actions/checkout@v4
@@ -177,6 +182,8 @@ jobs:
177182
integration-test-mac-spark3:
178183
runs-on: macos-15
179184
timeout-minutes: 20
185+
env:
186+
SPARK_LOCAL_IP: localhost
180187
steps:
181188
- uses: actions/checkout@v4
182189
- uses: swift-actions/setup-swift@d10500c1ac8822132eebbd74c48c3372c71d7ff5
@@ -203,6 +210,7 @@ jobs:
203210
runs-on: macos-15
204211
timeout-minutes: 20
205212
env:
213+
SPARK_LOCAL_IP: localhost
206214
SPARK_ICEBERG_TEST_ENABLED: "true"
207215
steps:
208216
- uses: actions/checkout@v4

0 commit comments

Comments
 (0)