[SPARK-54120] Update assertGeneratedCRDMatchesHelmChart to include diff

### What changes were proposed in this pull request?

Enhance existing test assertGeneratedCRDMatchesHelmChart to give diff upon failure

### Why are the changes needed?

For better readability.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Self-contained upon success & manually tested failure scenario - e.g. remove some fields from CRD inhelm repo, the test would fail with message like

```
* What went wrong:
Execution failed for task ':spark-operator-api:assertGeneratedCRDMatchesHelmChart'.
> Generated CRD yaml does not match the staged version in Helm Chart, please keep the chart updated.

  === SparkApplication CRD Differences ===
  --- /dev/fd/63        2025-10-31 14:33:55
  +++ /dev/fd/62        2025-10-31 14:33:55
   -6355,11 +6355,6
           properties:
             currentAttemptSummary:
               properties:
  -              attemptInfo:
  -                properties:
  -                  id:
  -                    type: integer
  -                type: object
                 stateTransitionHistory:
                   additionalProperties:
                     properties:

  === SparkCluster CRD Differences ===
  --- /dev/fd/63        2025-10-31 14:33:55
  +++ /dev/fd/62        2025-10-31 14:33:55
   -7218,8 +7218,6
                         type: string
                       lastTransitionTime:
                         type: string
  -                    message:
  -                      type: string
                     type: object
                   type: object
               type: object
```

Which gives a more clear indication about the exact failure

### Was this patch authored or co-authored using generative AI tooling?

No

Closes #414 from jiangzho/crd_diff.

Authored-by: Zhou JIANG <[email protected]>
Signed-off-by: Peter Toth <[email protected]>

Author: jiangzho

spark-operator-api/build.gradle

@@ -89,8 +89,31 @@ tasks.register("assertGeneratedCRDMatchesHelmChart") {
         "${stagedCRDFileBase}sparkclusters.spark.apache.org-v1.yaml"
     ].execute().text.trim()
     if (generatedAppCRD != stagedAppCRD || generatedClusterCRD != stagedClusterCRD) {
-      throw new GradleException("Generated CRD yaml does not match the staged version in " +
-          "Helm Chart, please keep the chart updated.")
+      def errorMessage = new StringBuilder("Generated CRD yaml does not match the staged " +
+          "version in Helm Chart, please keep the chart updated.\n\n")
+
+      if (generatedAppCRD != stagedAppCRD) {
+        errorMessage.append("=== SparkApplication CRD Differences ===\n")
+        def appDiff = ["bash", "-c",
+                       "diff -u <(echo '${generatedAppCRD.replace("'", "'\\''")}' " +
+                           "| yq -P 'sort_keys(..)') <(echo '${stagedAppCRD.replace("'", "'\\''")}' " +
+                           "| yq -P 'sort_keys(..)')"]
+            .execute().text
+        errorMessage.append(appDiff ?: "Unable to generate diff\n")
+        errorMessage.append("\n")
+      }
+
+      if (generatedClusterCRD != stagedClusterCRD) {
+        errorMessage.append("=== SparkCluster CRD Differences ===\n")
+        def clusterDiff = ["bash", "-c",
+                           "diff -u <(echo '${generatedClusterCRD.replace("'", "'\\''")}' " +
+                               "| yq -P 'sort_keys(..)') <(echo '${stagedClusterCRD.replace("'", "'\\''")}' " +
+                               "| yq -P 'sort_keys(..)')"]
+            .execute().text
+        errorMessage.append(clusterDiff ?: "Unable to generate diff\n")
+      }
+
+      throw new GradleException(errorMessage.toString())
     }
   }
 }