You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/worker/create_spark_session.rst
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ Configuring Spark session
5
5
6
6
SyncMaster Worker creates `SparkSession <https://spark.apache.org/docs/latest/sql-getting-started.html#starting-point-sparksession>`_ for each Run.
7
7
8
-
By default, SparkSession is created with ``master=local``, including all required .jar packages for DB/FileSystem types, and limited by transfer resources.
8
+
By default, SparkSession is created with ``spark.master: local``, including all required .jar packages for DB/FileSystem types, and limited by transfer resources.
9
9
10
10
Custom Spark session configuration
11
11
----------------------------------
@@ -23,7 +23,7 @@ It is possible to alter default `Spark Session configuration <https://spark.apac
23
23
spark.sql.pyspark.jvmStacktrace.enabled: true
24
24
spark.ui.enabled: false
25
25
26
-
For example, to use SyncMaster on Spark + Kubernetes, you can use worker image for Spark executor containers:
26
+
For example, to use SyncMaster on Spark-on-K8s, you can use worker image for Spark executor containers:
27
27
28
28
.. code-block:: yaml
29
29
:caption: config.yml
@@ -39,6 +39,10 @@ For example, to use SyncMaster on Spark + Kubernetes, you can use worker image f
0 commit comments