[SPARK-51075][K8S] Use container resource for HPA

### What changes were proposed in this pull request?

This PR aims to use `container resource` for HPA instead of `pod resource`.

### Why are the changes needed?

Since K8s 1.30, `Container resource based pod autoscaling` [KEP-1610](https://kep.k8s.io/1610) becomes `Stable`.
- https://v1-30.docs.kubernetes.io/blog/2024/03/12/kubernetes-1-30-upcoming-changes/

`container` metrics is more accurate the `pod` metric because Spark Worker pods frequently have more containers for logging and monitoring.

### Does this PR introduce _any_ user-facing change?

No, this is not released yet.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#156 from dongjoon-hyun/SPARK-51075.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

Author: dongjoon-hyun

examples/cluster-with-hpa-template.yaml

@@ -39,9 +39,10 @@ spec:
                 memory: "3Gi"
     horizontalPodAutoscalerSpec:
       metrics:
-      - type: Resource
-        resource:
+      - type: ContainerResource
+        containerResource:
           name: cpu
+          container: worker
           target:
             type: Utilization
             averageUtilization: 10

spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java

@@ -307,14 +307,15 @@ private static Optional<HorizontalPodAutoscaler> buildHorizontalPodAutoscaler(
           new HorizontalPodAutoscalerSpecBuilder()
               .addToMetrics(
                   new MetricSpecBuilder()
-                      .withType("Resource")
-                      .withNewResource()
+                      .withType("ContainerResource")
+                      .withNewContainerResource()
                       .withName("cpu")
+                      .withContainer("worker")
                       .withNewTarget()
                       .withType("Utilization")
                       .withAverageUtilization(30)
                       .endTarget()
-                      .endResource()
+                      .endContainerResource()
                       .build())
               .withNewBehavior()
               .withNewScaleUp()

spark-submission-worker/src/test/java/org/apache/spark/k8s/operator/SparkClusterResourceSpecTest.java

@@ -304,5 +304,7 @@ void testHorizontalPodAutoscaler() {
     assertEquals("4.0.0", hpa.getMetadata().getLabels().get(LABEL_SPARK_VERSION_NAME));
     assertEquals(1, hpa.getSpec().getMinReplicas());
     assertEquals(3, hpa.getSpec().getMaxReplicas());
+    assertEquals(1, hpa.getSpec().getMetrics().size());
+    assertEquals("worker", hpa.getSpec().getMetrics().get(0).getContainerResource().getContainer());
   }
 }