[SPARK-52292] Use super-linter for markdown files

### What changes were proposed in this pull request?

This PR aims to apply `super-linter` for markdown files.

### Why are the changes needed?

For consistency.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs with the newly added `super-linter` test.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#224 from dongjoon-hyun/SPARK-52292.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>

Author: dongjoon-hyun

.github/workflows/build_and_test.yml

@@ -147,6 +147,14 @@ jobs:
     steps:
       - name: Checkout repository
         uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Super-Linter
+        uses: super-linter/super-linter@12150456a73e248bdc94d0794898f94e23127c88
+        env:
+          DEFAULT_BRANCH: main
+          VALIDATE_MARKDOWN: true
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       - name: Set up JDK 17
         uses: actions/setup-java@v4
         with:

.markdownlint.yaml

@@ -0,0 +1,18 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+MD013: false

.markdownlintignore

@@ -0,0 +1,18 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+docs/config_properties.md

README.md

@@ -12,13 +12,14 @@ aims to extend K8s resource manager to manage Apache Spark applications via
 ## Install Helm Chart
 
 Apache Spark provides a Helm Chart.
+
 - <https://apache.github.io/spark-kubernetes-operator/>
 - <https://artifacthub.io/packages/helm/spark-kubernetes-operator/spark-kubernetes-operator/>
 
-```
-$ helm repo add spark-kubernetes-operator https://apache.github.io/spark-kubernetes-operator
-$ helm repo update
-$ helm install spark-kubernetes-operator spark-kubernetes-operator/spark-kubernetes-operator
+```bash
+helm repo add spark-kubernetes-operator https://apache.github.io/spark-kubernetes-operator
+helm repo update
+helm install spark-kubernetes-operator spark-kubernetes-operator/spark-kubernetes-operator
 ```
 
 ## Building Spark K8s Operator
@@ -27,25 +28,25 @@ Spark K8s Operator is built using Gradle.
 To build, run:
 
 ```bash
-$ ./gradlew build -x test
+./gradlew build -x test
 ```
 
 ## Running Tests
 
 ```bash
-$ ./gradlew build
+./gradlew build
 ```
 
 ## Build Docker Image
 
 ```bash
-$ ./gradlew buildDockerImage
+./gradlew buildDockerImage
 ```
 
-## Install Helm Chart
+## Install Helm Chart from the source code
 
 ```bash
-$ helm install spark -f build-tools/helm/spark-kubernetes-operator/values.yaml build-tools/helm/spark-kubernetes-operator/
+helm install spark -f build-tools/helm/spark-kubernetes-operator/values.yaml build-tools/helm/spark-kubernetes-operator/
 ```
 
 ## Run Spark Pi App
@@ -97,14 +98,14 @@ sparkcluster.spark.apache.org "prod" deleted
 
 ## Run Spark Pi App on Apache YuniKorn scheduler
 
-If you have not yet done so, follow [YuniKorn docs](https://yunikorn.apache.org/docs/#install) to install the latest version: 
+If you have not yet done so, follow [YuniKorn docs](https://yunikorn.apache.org/docs/#install) to install the latest version:
 
 ```bash
-$ helm repo add yunikorn https://apache.github.io/yunikorn-release
+helm repo add yunikorn https://apache.github.io/yunikorn-release
 
-$ helm repo update
+helm repo update
 
-$ helm install yunikorn yunikorn/yunikorn --namespace yunikorn --version 1.6.3 --create-namespace --set embedAdmissionController=false
+helm install yunikorn yunikorn/yunikorn --namespace yunikorn --version 1.6.3 --create-namespace --set embedAdmissionController=false
 ```
 
 Submit a Spark app to YuniKorn enabled cluster:
@@ -134,7 +135,7 @@ sparkapplication.spark.apache.org "pi-on-yunikorn" deleted
 
 Check the existing Spark applications and clusters. If exists, delete them.
 
-```
+```bash
 $ kubectl get sparkapp
 No resources found in default namespace.
 
@@ -144,12 +145,12 @@ No resources found in default namespace.
 
 Remove HelmChart and CRDs.
 
-```
-$ helm uninstall spark-kubernetes-operator
+```bash
+helm uninstall spark-kubernetes-operator
 
-$ kubectl delete crd sparkapplications.spark.apache.org
+kubectl delete crd sparkapplications.spark.apache.org
 
-$ kubectl delete crd sparkclusters.spark.apache.org
+kubectl delete crd sparkclusters.spark.apache.org
 ```
 
 ## Contributing

docs/architecture.md

@@ -23,10 +23,10 @@ under the License.
 deployment lifecycle of Spark applications and clusters. The Operator can be installed on Kubernetes
 cluster(s) using Helm. In most production environments it is typically deployed in a designated
 namespace and controls Spark workload in one or more managed namespaces.
-Spark Operator enables user to describe Spark application(s) or cluster(s) as 
-[Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). 
+Spark Operator enables user to describe Spark application(s) or cluster(s) as
+[Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/).
 
-The Operator continuously tracks events related to the Spark custom resources in its reconciliation 
+The Operator continuously tracks events related to the Spark custom resources in its reconciliation
 loops:
 
 For SparkApplications:
@@ -43,39 +43,39 @@ For SparkClusters:
 * Operator releases all Spark-cluster owned resources to cluster upon failure
 
 The Operator is built with the [Java Operator SDK](https://javaoperatorsdk.io/) for
-launching Spark deployments and submitting jobs under the hood. It also uses 
+launching Spark deployments and submitting jobs under the hood. It also uses
 [fabric8](https://fabric8.io/) client to interact with Kubernetes API Server.
 
 ## Application State Transition
 
-[<img src="resources/application_state_machine.png">](resources/application_state_machine.png)
+[![Application State Transition](resources/application_state_machine.png)](resources/application_state_machine.png)
 
 * Spark applications are expected to run from submitted to succeeded before releasing resources
 * User may configure the app CR to time-out after given threshold of time if it cannot reach healthy
-  state after given threshold. The timeout can be configured for different lifecycle stages, 
+  state after given threshold. The timeout can be configured for different lifecycle stages,
   when driver starting and when requesting executor pods. To update the default threshold,  
-  configure `.spec.applicationTolerations.applicationTimeoutConfig` for the application.        
-* K8s resources created for an application would be deleted as the final stage of the application 
+  configure `.spec.applicationTolerations.applicationTimeoutConfig` for the application.
+* K8s resources created for an application would be deleted as the final stage of the application
   lifecycle by default. This is to ensure resource quota release for completed applications.  
-* It is also possible to retain the created k8s resources for debug or audit purpose. To do so,   
-  user may set `.spec.applicationTolerations.resourceRetainPolicy` to `OnFailure` to retain 
-  resources upon application failure, or set to `Always` to retain resources regardless of 
+* It is also possible to retain the created k8s resources for debug or audit purpose. To do so,
+  user may set `.spec.applicationTolerations.resourceRetainPolicy` to `OnFailure` to retain
+  resources upon application failure, or set to `Always` to retain resources regardless of
   application final state.
-    - This controls the behavior of k8s resources created by Operator for the application, including
-      driver pod, config map, service, and PVC(if enabled). This does not apply to resources created 
+  * This controls the behavior of k8s resources created by Operator for the application, including
+      driver pod, config map, service, and PVC(if enabled). This does not apply to resources created
       by driver (for example, executor pods). User may configure SparkConf to
-      include `spark.kubernetes.executor.deleteOnTermination` for executor retention. Please refer 
+      include `spark.kubernetes.executor.deleteOnTermination` for executor retention. Please refer
       [Spark docs](https://spark.apache.org/docs/latest/running-on-kubernetes.html) for details.
-    - The created k8s resources have `ownerReference` to their related `SparkApplication` custom
+  * The created k8s resources have `ownerReference` to their related `SparkApplication` custom
       resource, such that they could be garbage collected when the `SparkApplication` is deleted.
-    - Please be advised that k8s resources would not be retained if the application is configured to
-      restart. This is to avoid resource quota usage increase unexpectedly or resource conflicts 
+  * Please be advised that k8s resources would not be retained if the application is configured to
+      restart. This is to avoid resource quota usage increase unexpectedly or resource conflicts
       among multiple attempts.
 
 ## Cluster State Transition
 
-[<img src="resources/cluster_state_machine.png">](resources/application_state_machine.png)
+[![Cluster State Transition](resources/application_state_machine.png)](resources/application_state_machine.png)
 
 * Spark clusters are expected to be always running after submitted.
-* Similar to Spark applications, K8s resources created for a cluster would be deleted as the final 
+* Similar to Spark applications, K8s resources created for a cluster would be deleted as the final
   stage of the cluster lifecycle by default.

docs/configuration.md

@@ -29,20 +29,20 @@ Spark Operator supports different ways to configure the behavior:
   files](../build-tools/helm/spark-kubernetes-operator/values.yaml).
 * **System Properties** : when provided as system properties (e.g. via -D options to the
   operator JVM), it overrides the values provided in property file.
-* **Hot property loading** : when enabled, a 
-  [configmap](https://kubernetes.io/docs/concepts/configuration/configmap/) would be created with 
-  the operator in the same namespace. Operator can monitor updates performed on the configmap. Hot 
+* **Hot property loading** : when enabled, a
+  [configmap](https://kubernetes.io/docs/concepts/configuration/configmap/) would be created with
+  the operator in the same namespace. Operator can monitor updates performed on the configmap. Hot
   properties reloading takes higher precedence comparing with default properties override.
-    - An example use case: operator use hot properties to figure the list of namespace(s) to
+  * An example use case: operator use hot properties to figure the list of namespace(s) to
       operate Spark applications. The hot properties config map can be updated and
       maintained by user or additional microservice to tune the operator behavior without
       rebooting it.
-    - Please be advised that not all properties can be hot-loaded and honored at runtime.
+  * Please be advised that not all properties can be hot-loaded and honored at runtime.
       Refer the list of [supported properties](./config_properties.md) for more details.
 
 To enable hot properties loading, update the **helm chart values file** with
 
-```
+```yaml
 operatorConfiguration:
   spark-operator.properties: |+
     spark.operator.dynamic.config.enabled=true
@@ -60,18 +60,18 @@ the [Dropwizard Metrics Library](https://metrics.dropwizard.io/4.2.25/). Note th
 does not have Spark UI, MetricsServlet
 and PrometheusServlet from org.apache.spark.metrics.sink package are not supported. If you are
 interested in Prometheus metrics exporting, please take a look at below
-section [Forward Metrics to Prometheus](#Forward-Metrics-to-Prometheus)
+section [Forward Metrics to Prometheus](#forward-metrics-to-prometheus)
 
 ### JVM Metrics
 
 Spark Operator collects JVM metrics
 via [Codahale JVM Metrics](https://javadoc.io/doc/com.codahale.metrics/metrics-jvm/latest/index.html)
 
-- BufferPoolMetricSet
-- FileDescriptorRatioGauge
-- GarbageCollectorMetricSet
-- MemoryUsageGaugeSet
-- ThreadStatesGaugeSet
+* BufferPoolMetricSet
+* FileDescriptorRatioGauge
+* GarbageCollectorMetricSet
+* MemoryUsageGaugeSet
+* ThreadStatesGaugeSet
 
 ### Kubernetes Client Metrics
 
@@ -81,15 +81,15 @@ via [Codahale JVM Metrics](https://javadoc.io/doc/com.codahale.metrics/metrics-j
 | kubernetes.client.http.response                           | Meter      | Tracking the rates of HTTP response from the Kubernetes API Server                                                       |
 | kubernetes.client.http.response.failed                    | Meter      | Tracking the rates of HTTP requests which have no response from the Kubernetes API Server                                |
 | kubernetes.client.http.response.latency.nanos             | Histograms | Measures the statistical distribution of HTTP response latency from the Kubernetes API Server                            |
-| kubernetes.client.http.response.<ResponseCode>            | Meter      | Tracking the rates of HTTP response based on response code from the Kubernetes API Server                                |
-| kubernetes.client.http.request.<RequestMethod>            | Meter      | Tracking the rates of HTTP request based type of method to the Kubernetes API Server                                     |
+| kubernetes.client.http.response.`ResponseCode`            | Meter      | Tracking the rates of HTTP response based on response code from the Kubernetes API Server                                |
+| kubernetes.client.http.request.`RequestMethod`            | Meter      | Tracking the rates of HTTP request based type of method to the Kubernetes API Server                                     |
 | kubernetes.client.http.response.1xx                       | Meter      | Tracking the rates of HTTP Code 1xx responses (informational) received from the Kubernetes API Server per response code. |
 | kubernetes.client.http.response.2xx                       | Meter      | Tracking the rates of HTTP Code 2xx responses (success) received from the Kubernetes API Server per response code.       |
 | kubernetes.client.http.response.3xx                       | Meter      | Tracking the rates of HTTP Code 3xx responses (redirection) received from the Kubernetes API Server per response code.   |
 | kubernetes.client.http.response.4xx                       | Meter      | Tracking the rates of HTTP Code 4xx responses (client error) received from the Kubernetes API Server per response code.  |
 | kubernetes.client.http.response.5xx                       | Meter      | Tracking the rates of HTTP Code 5xx responses (server error) received from the Kubernetes API Server per response code.  |
-| kubernetes.client.<ResourceName>.<Method>                 | Meter      | Tracking the rates of HTTP request for a combination of one Kubernetes resource and one http method                      |
-| kubernetes.client.<NamespaceName>.<ResourceName>.<Method> | Meter      | Tracking the rates of HTTP request for a combination of one namespace-scoped Kubernetes resource and one http method     |
+| kubernetes.client.`ResourceName`.`Method`                 | Meter      | Tracking the rates of HTTP request for a combination of one Kubernetes resource and one http method                      |
+| kubernetes.client.`NamespaceName`.`ResourceName`.`Method` | Meter      | Tracking the rates of HTTP request for a combination of one namespace-scoped Kubernetes resource and one http method     |
 
 ### Forward Metrics to Prometheus
 
@@ -141,4 +141,4 @@ kubectl port-forward --address 0.0.0.0 pod/prometheus-server-654bc74fc9-8hgkb  8
 
 open your browser with address `localhost:8080`. Click on Status Targets tab, you should be able
 to find target as below.
-[<img src="resources/prometheus.png">](resources/prometheus.png)
+[![Prometheus](resources/prometheus.png)](resources/prometheus.png)

docs/operations.md

@@ -17,15 +17,17 @@ specific language governing permissions and limitations
 under the License.
 -->
 
-### Compatibility
+# Operations
+
+## Compatibility
 
 - Java 17, 21 and 24
 - Kubernetes version compatibility:
-    + k8s version >= 1.30 is recommended. Operator attempts to be API compatible as possible, but
+  - k8s version >= 1.30 is recommended. Operator attempts to be API compatible as possible, but
       patch support will not be performed on k8s versions that reached EOL.
 - Spark versions 3.5 or above.
 
-### Spark Application Namespaces
+## Spark Application Namespaces
 
 By default, Spark applications are created in the same namespace as the operator deployment.
 You many also configure the chart deployment to add necessary RBAC resources for
@@ -38,7 +40,7 @@ in `values.yaml`) for the Helm chart.
 
 To override single parameters you can use `--set`, for example:
 
-```
+```bash
 helm install --set image.repository=<my_registory>/spark-kubernetes-operator \
    -f build-tools/helm/spark-kubernetes-operator/values.yaml \
   build-tools/helm/spark-kubernetes-operator/
@@ -47,7 +49,7 @@ helm install --set image.repository=<my_registory>/spark-kubernetes-operator \
 You can also provide multiple custom values file by using the `-f` flag, the latest takes
 higher precedence:
 
-```
+```bash
 helm install spark-kubernetes-operator \
    -f build-tools/helm/spark-kubernetes-operator/values.yaml \
    -f my_values.yaml \