Standalone Spark - better Kubernetes integration

[banzo]

New versions of Spark have better Kubernetes integration: https://kubernetes.io/blog/2018/03/apache-spark-23-with-native-kubernetes/

This sample chart provides some information on the deeper integration, but also highlights possible limitations, this need to be studied a bit deeper.

Thanks to @terence-bigtt for the heads up.

[banzo]

We might want to implement this as an operator: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator