Skip to content

Commit 7ae3f18

Browse files
committed
fix: use {} if sparkConf not set
-e Signed-off-by: machichima <nary12321@gmail.com>
1 parent 989eb67 commit 7ae3f18

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

plugins/flytekit-spark/flytekitplugins/spark/connector.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ def _get_databricks_job_spec(task_template: TaskTemplate) -> dict:
4040
if not new_cluster.get("docker_image"):
4141
new_cluster["docker_image"] = {"url": container.image}
4242
if not new_cluster.get("spark_conf"):
43-
new_cluster["spark_conf"] = custom["sparkConf"]
43+
new_cluster["spark_conf"] = custom.get("sparkConf", {})
4444
if not new_cluster.get("spark_env_vars"):
4545
new_cluster["spark_env_vars"] = {k: v for k, v in envs.items()}
4646
else:

0 commit comments

Comments
 (0)