MLE-24826 Bumping Spring, Spark, and ml-gradle

[rjrudin]

And also trying out mlWaitTillReady, plus some general Jenkins cleanup.

[github-actions]

Copyright Validation Results
Total: 8 | Passed: 3 | Failed: 0 | Skipped: 5 | at: 2025-10-22 15:06:32 UTC | commit: 39f7917

⏭️ Skipped (Excluded) Files

• Jenkinsfile
• build.gradle
• gradle.properties
• marklogic-spark-connector/build.gradle
• test-app/build.gradle

✅ Valid Files

• marklogic-spark-connector/src/main/java/com/marklogic/spark/reader/CustomLongOffset.java
• marklogic-spark-connector/src/main/java/com/marklogic/spark/reader/customcode/CustomCodeMicroBatchStream.java
• marklogic-spark-connector/src/main/java/com/marklogic/spark/reader/optic/OpticMicroBatchStream.java

✅ All files have valid copyright headers!

[Copilot]

Pull Request Overview

This PR upgrades Spring and Spark dependencies to newer versions and integrates ml-gradle 6.2-SNAPSHOT. The changes include version bumps for Spark (4.1.0-preview2), Spring dependencies via ml-app-deployer and marklogic-junit5 upgrades, and Jenkins pipeline improvements using the new mlWaitTillReady task.

• Upgraded Spark from 4.1.0-preview1 to 4.1.0-preview2
• Updated ml-gradle from 6.0.1 to 6.2-SNAPSHOT with corresponding dependency updates
• Refactored Jenkins pipeline to use mlWaitTillReady and consolidated cleanup logic

Reviewed Changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

File	Description
test-app/build.gradle	Migrated from plugin DSL to buildscript block to use ml-gradle 6.2-SNAPSHOT
marklogic-spark-connector/build.gradle	Updated dependency versions including Spark, JUnit, ml-app-deployer, and removed dependency exclusions
gradle.properties	Bumped Spark version to 4.1.0-preview2
Jenkinsfile	Refactored test execution with mlWaitTillReady, extracted teardown logic, and removed hardcoded sleep

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Double dollar sign $$JAVA_HOME will not correctly expand the environment variable. Should be $JAVA_HOME (single dollar sign).

Suggested change

	export PATH=$$JAVA_HOME/bin:$PATH
	export PATH=$JAVA_HOME/bin:$PATH

[sonarqube-progress-marklogic]

Quality Gate failed

Failed conditions
1 New issue
11.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE