MLE-24402 Trying out Spark 4.1.0 preview1 #559
Conversation
rjrudin
requested review from
BillFarber,
anu3990 and
stevebio
as code owners
|
Copyright Validation Results ⏭️ Skipped (Excluded) Files
✅ All files have valid copyright headers! |
There was a problem hiding this comment.
Pull Request Overview
This PR upgrades the Spark dependency from version 4.0.1 to 4.1.0-preview1 and refactors the build configuration to simplify dependency management by removing the custom shadowDependencies configuration in favor of standard Gradle configurations.
- Updates Spark version to 4.1.0-preview1 which includes Jackson 2.19.0 to reduce CVE concerns
- Removes custom
shadowDependenciesconfiguration and converts dependencies to standardimplementationandapiconfigurations - Updates shadow plugin version and disables shadow jar publishing to Maven repositories
Reviewed Changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| marklogic-spark-connector/build.gradle | Refactors dependency management by removing shadowDependencies configuration, updates shadow plugin, and converts dependencies to standard Gradle configurations |
| gradle.properties | Updates Spark version from 4.0.1 to 4.1.0-preview1 |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| // Using 'api' so that Flux does not need to declare this as well. | ||
| api("com.marklogic:marklogic-client-api:7.2.0") { |
There was a problem hiding this comment.
Converting from shadowDependencies to 'api' exposes this dependency transitively to consumers. This is a breaking change in the API surface. Consider using 'implementation' instead to maintain encapsulation, unless transitive exposure is intentional.
| // Using 'api' so that Flux does not need to declare this as well. | |
| api("com.marklogic:marklogic-client-api:7.2.0") { | |
| // Using 'implementation' to avoid exposing this dependency transitively to consumers. | |
| implementation("com.marklogic:marklogic-client-api:7.2.0") { |
This is tempting to start using now because it requires fewer dependency alterations to minimize CVEs. Tests all appear to be fine. And 4.1.0 is using Jackson 2.19.0, which avoids a lot of messing around.
rjrudin
force-pushed
the
feature/4.1-preview
branch
from
5bfe11c to
90822e4
Compare
|
This is tempting to start using now because it requires fewer dependency alterations to minimize CVEs. Tests all appear to be fine.
And 4.1.0 is using Jackson 2.19.0, which avoids a lot of messing around.