[SPARK-51520][K8S] Use JRE to reduce size of Docker image

### What changes were proposed in this pull request?

* Switch to Azul JRE to save ~100 megabytes on the final runtime image
* Add a Buildkit cache mount for the Gradle cache to speed up local Docker builds during development

### Why are the changes needed?

The image is currently quite large at ~500 megabytes

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Tested locally

### Was this patch authored or co-authored using generative AI tooling?

No

Closes #167 from jacobsalway/use-jre-to-reduce-docker-image-size.

Authored-by: Jacob Salway <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

Author: jacobsalway

build-tools/docker/Dockerfile

@@ -15,13 +15,18 @@
 # limitations under the License.
 #
 FROM gradle:8.13-jdk17-noble AS builder
+
 WORKDIR /app
+
 COPY . .
-RUN ./gradlew clean build -x check
 
-FROM azul/zulu-openjdk:21
+RUN --mount=type=cache,target=/home/gradle/.gradle/caches gradle --no-daemon clean build -x check
+
+FROM azul/zulu-openjdk:21-jre
+
 ARG APP_VERSION=0.1.0-SNAPSHOT
 ARG SPARK_UID=185
+
 LABEL org.opencontainers.image.authors="Apache Spark project <[email protected]>"
 LABEL org.opencontainers.image.licenses="Apache-2.0"
 LABEL org.opencontainers.image.ref.name="Apache Spark Kubernetes Operator"
@@ -32,14 +37,16 @@ ENV SPARK_OPERATOR_WORK_DIR=/opt/spark-operator/operator
 ENV SPARK_OPERATOR_JAR=spark-kubernetes-operator.jar
 
 WORKDIR $SPARK_OPERATOR_WORK_DIR
+
 RUN groupadd --system --gid=$SPARK_UID spark && \
     useradd --system --home-dir $SPARK_OPERATOR_HOME --uid=$SPARK_UID --gid=spark spark && \
     chown -R spark:spark $SPARK_OPERATOR_HOME
 
 COPY --from=builder --chown=spark:spark /app/spark-operator/build/libs/spark-kubernetes-operator-$APP_VERSION-all.jar $SPARK_OPERATOR_JAR
 COPY --from=builder --chown=spark:spark /app/build-tools/docker/docker-entrypoint.sh .
 
-
 USER spark
+
 ENTRYPOINT ["/opt/spark-operator/operator/docker-entrypoint.sh"]
+
 CMD ["help"]