MLE-24402 Cleaning up projects and contrib

Author: rjrudin

.copyrightconfig

@@ -11,4 +11,4 @@ startyear: 2023
 #   - Dotfiles already skipped automatically
 # Enable by removing the leading '# ' from the next line and editing values.
 # filesexcluded: third_party/*, docs/generated/*.md, assets/*.png, scripts/temp_*.py, vendor/lib.js
-filesexcluded: .github/*, README.md, Jenkinsfile, gradle/*, docker-compose.yml, *.gradle, gradle.properties, gradlew, gradlew.bat, **/test/resources/**
+filesexcluded: .github/*, README.md, CONTRIBUTING.md, Jenkinsfile, gradle/*, docker-compose.yml, *.gradle, gradle.properties, gradlew, gradlew.bat, **/test/resources/**, docs/**

CONTRIBUTING.md

@@ -1,12 +1,7 @@
 This guide covers how to develop and test this project. It assumes that you have cloned this repository to your local
 workstation.
 
-**You must use Java 17 for developing, testing, and building this project**, even though the connector supports
-running on Java 11. For users, Java 17 is only required if using the splitting and embedding features, as those
-depend on a third party module that requires Java 17.
-
-**You also need Java 11 installed** so that the subprojects in this repository that require Java 11 have access to a 
-Java 11 SDK. [sdkman](https://sdkman.io/) is highly recommend for installing multiple JDKs.
+**You must use Java 17 for developing, testing, and building this project**.
 
 # Setup
 
@@ -44,6 +39,12 @@ To run the tests against the test application, run the following Gradle task:
 
     ./gradlew test
 
+To also get code coverage, run:
+
+    ./gradlew test jacocoTestReport
+
+The code coverage report will be in `marklogic-spark-connector/build/reports/jacoco/test/html/index.html`.
+
 **To run the tests in Intellij**, you must configure your JUnit template to include a few JVM args:
 
 1. Go to Run -> Edit Configurations.

Jenkinsfile

@@ -15,7 +15,7 @@ def runtests(String javaVersion){
     ./gradlew -i mlDeploy
     echo "Loading data a second time to try to avoid Optic bug with duplicate rows being returned."
     ./gradlew -i mlLoadData
-    ./gradlew clean testCodeCoverageReport || true
+    ./gradlew clean test jacocoTestReport || true
   '''
   junit '**/build/**/*.xml'
 }

build.gradle

@@ -8,7 +8,7 @@ sonar {
   properties {
     property "sonar.projectKey", "marklogic-spark"
     property "sonar.host.url", "http://localhost:9000"
-    property "sonar.coverage.jacoco.xmlReportPaths", "code-coverage-report/build/reports/jacoco/testCodeCoverageReport/testCodeCoverageReport.xml"
+    property "sonar.coverage.jacoco.xmlReportPaths", "marklogic-spark-connector/build/reports/jacoco/test/jacocoTestReport.xml"
   }
 }
 

code-coverage-report/build.gradle

@@ -12,11 +12,10 @@ from any data source that Spark supports and then writing it to MarkLogic.
 
 The connector has the following system requirements:
 
-* Apache Spark 3.5.6 is recommended, but earlier versions of Spark 3.5.x, 3.4.x, and 3.3.x should work as well. When choosing 
-[a Spark distribution](https://spark.apache.org/downloads.html), you must select a distribution that uses Scala 2.12 and not Scala 2.13.
+* Apache Spark 4.0 or higher. 
+* If using Java to run Spark, Spark 4 requires Java 17 or higher.
 * For writing data, MarkLogic 9.0-9 or higher.
 * For reading data, MarkLogic 10.0-9 or higher.
-* If using Java to run Spark, Java 11 or higher is required.
 
 In addition, if your MarkLogic cluster has multiple hosts in it, it is highly recommended to put a load balancer in front
 of your cluster and have the MarkLogic Spark connector connect through the load balancer. 

docs/index.md

@@ -1,4 +1,7 @@
+// The connector code was previously split across multiple subprojects to allow for different Java
+// requirements. That separation is no longer needed now that the connector requires Spark 4 and thus
+// requires Java 17. The marklogic-spark-connector subproject is being kept for now in case there's
+// a need for another subproject in the future. And having the test-app in a separate subproject is
+// cleaner than including it in the base project. 
 include "marklogic-spark-connector"
-//include "tests"
 include "test-app"
-include "code-coverage-report"