[SPARK-51538] Add spark-connect-server-with-spark-cluster.yaml example

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to add an example to launch Spark Connect Server pod which uses the existing Spark Cluster.

Why are the changes needed?

Since Spark Connect Server pod is launched outside of Spark Cluster, we can expose the service port more easily.

Does this PR introduce any user-facing change?

No. This is a new example.

How was this patch tested?

Manual testing.

Launch Spark K8s Operator, Spark Cluster, and Spark Connect Server sequentially.

$ helm install spark-kubernetes-operator \
https://nightlies.apache.org/spark/charts/spark-kubernetes-operator-0.1.0-SNAPSHOT.tgz

$ kubectl apply -f examples/prod-cluster-with-three-workers.yaml

$ kubectl apply -f examples/spark-connect-server-with-spark-cluster.yaml

After exporting the Spark Master and Spark Connect Server ports, connect with pyspark client.

$ bin/pyspark --remote sc://localhost:15002

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

Thank you, @viirya ! Merged to main.