Skip to content

Commit 0771de6

Browse files
authored
doc: update macos standalone spark start instructions (#2103)
1 parent 41c69d5 commit 0771de6

File tree

1 file changed

+11
-10
lines changed

1 file changed

+11
-10
lines changed

docs/source/contributor-guide/benchmarking_macos.md

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -61,16 +61,6 @@ Start Spark in standalone mode:
6161
$SPARK_HOME/sbin/start-master.sh
6262
```
6363

64-
Note: For Apache Spark distributions installed using `brew` tool, it may happen there is no `$SPARK_HOME/sbin` folder on your machine. However it is still possible to start Apache Spark master by running command
65-
```shell
66-
$SPARK_HOME/bin/spark-class org.apache.spark.deploy.master.Master
67-
```
68-
69-
Once master has started, look for output to find Master endpoint URI, like
70-
```shell
71-
INFO Master: Starting Spark master at spark://192.168.4.142:7078
72-
```
73-
7464
Set `SPARK_MASTER` env var (host name will need to be edited):
7565

7666
```shell
@@ -82,6 +72,17 @@ export SPARK_MASTER=spark://Rustys-MacBook-Pro.local:7077
8272
$SPARK_HOME/sbin/start-worker.sh $SPARK_MASTER
8373
```
8474

75+
### Start local Apache Spark cluster using `spark-class`
76+
For Apache Spark distributions installed using `brew` tool, it may happen there is no `$SPARK_HOME/sbin` folder on your machine.
77+
In order to start local Apache Spark cluster on `localhost:7077` port, run:
78+
```shell
79+
$SPARK_HOME/bin/spark-class org.apache.spark.deploy.master.Master --host 127.0.0.1 --port 7077 --webui-port 8080
80+
```
81+
82+
Once master has started, in separate console start the worker referring the spark master uri on `localhost:7077`
83+
```shell
84+
$SPARK_HOME/bin/spark-class org.apache.spark.deploy.worker.Worker --cores 8 --memory 16G spark://localhost:7077
85+
```
8586

8687
## Run Spark Benchmarks
8788

0 commit comments

Comments
 (0)