[SPARK-52253] Support both v1alpha1 and v1beta1

### What changes were proposed in this pull request?

This PR aims to support `v1alpha1` and `v1beta1`.
- Both `v1alpha1` and `v1beta1` will be accepted and be stored as `v1beta1`.
- Since the CRD generator always write a single version, we cannot use it directly from now. We should keep all previous versions in the CRD files inevitably.
- It's also required to protect these CRDs from accidental and breaking changes from `Beta`.

The added CRD files look like the following. Please note that `v1alpha` is `storage: false` and `v1beta1` is `storage: true`.

```yaml
# ... APACHE HEADER ...
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: sparkapplications.spark.apache.org
spec:
  group: spark.apache.org
  names:
    kind: SparkApplication
    plural: sparkapplications
    shortNames:
      - sparkapp
    singular: sparkapplication
  scope: Namespaced
  versions:
    - name: v1alpha1
      storage: false
       ...
    - name: v1beta1
      storage: true
       ...
```

### Why are the changes needed?

- To support a smooth migration.
- To be clear, there is no schema change from 0.2.0 to 0.3.0 so far. Only, versions are changed.

### Does this PR introduce _any_ user-facing change?

No behavior change.

### How was this patch tested?

Pass the CIs.

After manual installation,

```
$ kubectl get crds sparkapplications.spark.apache.org -oyaml | yq '.spec.versions[].name'
v1alpha1
v1beta1

$ kubectl get crds sparkapplications.spark.apache.org -oyaml | yq .status.storedVersions
- v1beta1
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #220 from dongjoon-hyun/SPARK-52253.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

Author: dongjoon-hyun

.github/workflows/build_and_test.yml

@@ -118,7 +118,6 @@ jobs:
         run: |
           eval $(minikube docker-env)
           ./gradlew buildDockerImage
-          ./gradlew spark-operator-api:relocateGeneratedCRD
           helm install spark-kubernetes-operator --create-namespace -f build-tools/helm/spark-kubernetes-operator/values.yaml build-tools/helm/spark-kubernetes-operator/
           helm test spark-kubernetes-operator
           # Use remote host' s docker image
@@ -132,7 +131,6 @@ jobs:
         run: |
           eval $(minikube docker-env)
           ./gradlew buildDockerImage
-          ./gradlew spark-operator-api:relocateGeneratedCRD
           helm install spark-kubernetes-operator --create-namespace -f \
           build-tools/helm/spark-kubernetes-operator/values.yaml -f \
           tests/e2e/helm/dynamic-config-values.yaml \

.github/workflows/publish_snapshot_chart.yml

@@ -34,7 +34,7 @@ jobs:
         cache: 'gradle'
     - name: Build Operator
       run: |
-        ./gradlew build spark-operator-api:relocateGeneratedCRD -x check --no-daemon
+        ./gradlew build -x check --no-daemon
     - name: Build Chart
       env:
         DIR: 'charts'

.gitignore

@@ -6,7 +6,6 @@
 .vscode
 /lib/
 target/
-build-tools/helm/spark-kubernetes-operator/crds
 
 # Gradle Files #
 ################

README.md

@@ -45,8 +45,6 @@ $ ./gradlew buildDockerImage
 ## Install Helm Chart
 
 ```bash
-$ ./gradlew spark-operator-api:relocateGeneratedCRD
-
 $ helm install spark -f build-tools/helm/spark-kubernetes-operator/values.yaml build-tools/helm/spark-kubernetes-operator/
 ```