You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-53380] Add SPARK_LOCAL_IP env variable to MacOS-based integration test jobs
### What changes were proposed in this pull request?
This PR aims to recover MacOS-based integration tests by adding `SPARK_LOCAL_IP` environment variables like Apache Spark main repository.
### Why are the changes needed?
To recover MacOS-based CI failures. Currently, all MacOS-based integration test GitHub Action jobs are broken.
The root cause is that `SparkContext` fails due to `Invalid Spark URL` error like the following because `getHost` is `null` in Java.
```
$ jshell
jshell> new java.net.URI("spark://HeartbeatReceiversat12-bq147_186fa0a6-15fa-47f9-ad18-71676ed8dcea-4A992716D7F1.local:49174").getHost()
$1 ==> null
```
```
25/08/26 06:20:49 INFO SparkConnectServer: Starting Spark session.
25/08/26 06:20:50 INFO SparkContext: Running Spark version 4.1.0-preview1
25/08/26 06:20:50 INFO SparkContext: OS info Mac OS X, 15.5, aarch64
25/08/26 06:20:50 INFO SparkContext: Java version 21.0.8
...
25/08/26 06:20:50 INFO JettyUtils: Start Jetty 0.0.0.0:4040 for SparkUI
25/08/26 06:20:50 INFO Utils: Successfully started service 'SparkUI' on port 4040.
...
25/08/26 06:20:50 INFO Executor: Starting executor ID driver on host sat12-bq147_186fa0a6-15fa-47f9-ad18-71676ed8dcea-4A992716D7F1.local
...
25/08/26 06:20:50 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Invalid Spark URL: spark://HeartbeatReceiversat12-bq147_186fa0a6-15fa-47f9-ad18-71676ed8dcea-4A992716D7F1.local:49174
...
25/08/26 06:20:50 INFO SparkContext: SparkContext is stopping with exitCode 0 from main at NativeMethodAccessorImpl.java:0.
```
- https://github.com/apache/spark-connect-swift/actions/runs/17229744776/job/48881088121
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes#230 from dongjoon-hyun/SPARK_LOCAL_IP.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
0 commit comments