[SPARK-52253] Support both v1alpha1 and v1beta1

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to support v1alpha1 and v1beta1.

• Both v1alpha1 and v1beta1 will be accepted and be stored as v1beta1.
• Since the CRD generator always write a single version, we cannot use it directly from now. We should keep all previous versions in the CRD files inevitably.
• It's also required to protect these CRDs from accidental and breaking changes from Beta.

The added CRD files look like the following. Please note that v1alpha is storage: false and v1beta1 is storage: true.

# ... APACHE HEADER ...
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: sparkapplications.spark.apache.org
spec:
  group: spark.apache.org
  names:
    kind: SparkApplication
    plural: sparkapplications
    shortNames:
      - sparkapp
    singular: sparkapplication
  scope: Namespaced
  versions:
    - name: v1alpha1
      storage: false
       ...
    - name: v1beta1
      storage: true
       ...

Why are the changes needed?

• To support a smooth migration.
• To be clear, there is no schema change from 0.2.0 to 0.3.0 so far. Only, versions are changed.

Does this PR introduce any user-facing change?

No behavior change.

How was this patch tested?

Pass the CIs.

After manual installation,

$ kubectl get crds sparkapplications.spark.apache.org -oyaml | yq '.spec.versions[].name'
v1alpha1
v1beta1

$ kubectl get crds sparkapplications.spark.apache.org -oyaml | yq .status.storedVersions
- v1beta1

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

cc @viirya , @peter-toth , @jiangzho

[dongjoon-hyun]

Thank you, @viirya ! Merged to main.