This repository was archived by the owner on Jan 9, 2020. It is now read-only.
v2.2.0-kubernetes-0.4.0
Pre-release
Pre-release
·
22 commits
to branch-2.2-kubernetes
since this release
Major features and fixes in this release include:
- Support for HDFS locality.
- Added an option to use a secret to mount small files in driver and executors.
- Support for custom Kubernetes service account for the driver pod.
- Support for executor java options.
- Fixed conversion from MB to MiB in driver and executor memory specification.
- Added configuration properties for injecting arbitrary Kubernetes secrets into the driver and executors.
- Use a headless service to give a hostname to the driver (requiring Kubernetes DNS in the cluster).
- Improved docker image build/push flow.
- Added the ability to fail submission if submitter-local files are provided without the resource staging server URI.
- Added reference YAML files for RBAC configs for driver and shuffle service.
Deprecations and removals:
- Removed support for
spark.kubernetes.driver.labels
,spark.kubernetes.driver.annotations
,spark.kubernetes.executor.labels
,
spark.kubernetes.executor.annotations
which were deprecated. They have been superseded by new properties.
See documentation for details.