Skip to content

Make spark.home and spark.kubernetes.file.upload.path configurable. #165

@sudiptob2

Description

@sudiptob2

InsubmitArmadaSpark.sh script,

Currently /opt/spark and /opt/spark/bin/spark-submit are hard-coded. These paths should be configurable.
SPARK_HOME should be used to locate Spark binaries, falling back to /opt/spark if not set (e.g., ${SPARK_HOME:-/opt/spark}).

Similarly for --conf spark.kubernetes.file.upload.path=/tmp

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions