[SPARK-52836] Fix sparkapp.sh to let Spark driver determine pod memory

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to fix the benchmark script sparkapp.sh to let Spark Driver determine pod request memory.

Why are the changes needed?

Apache Spark driver has spark.driver.memory is 1g by default and adds overhead when it creates pod. So, we had better use Spark's built-in logic.

Does this PR introduce any user-facing change?

No, this is a benchmark script change.

How was this patch tested?

Manual run since the benchmark script is irrelevant to the CI.

$ cd tests/benchmark

# The default value is 1k. But, we need to use small value on laptop.
$ ./sparkapps.sh 50
CLEAN UP NAMESPACE FOR BENCHMARK
START BENCHMARK WITH 50 JOBS
FINISHED 50 JOBS IN 52 SECONDS.
DELETED 50 JOBS IN 16 SECONDS.

# While running the benchmark, we can check the memory.
$ kubectl get pod -l spark-role=driver -oyaml | grep memory | sort | uniq -c
 100           memory: 256Mi

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

cc @peter-toth

[dongjoon-hyun]

Thank you, @peter-toth . Merged to main.