[SPARK-51603][CONNECT][TESTS] Auto ignore tests that require starting SparkConnectServer when the directory assembly/target/scala-2.13/jars/ not exist

### What changes were proposed in this pull request?
After the merge of SPARK-48936 (#47402), the Maven module compilation order for the `connect-client-jvm` module was moved up to occur before the `assembly` build.

Since a portion of the tests in `connect-client-jvm` heavily rely on `SparkConnectServer` and related dependencies being collected into `assembly/target/scala-2.13/jars/`, it is necessary to build `assembly` before running `mvn test` for `connect-client-jvm`.

For example:

```
build/mvn clean install -DskipTests -Phive
build/mvn test -pl sql/connect/client/jvm
```

This sequence will result in successful tests.

However, if a full test suite is executed directly, such as:

```
build/mvn clean install -Phive
```

Failures similar to the following may occur:

```
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.044 s <<< FAILURE! -- in org.apache.spark.sql.JavaEncoderSuite
[ERROR] org.apache.spark.sql.JavaEncoderSuite -- Time elapsed: 0.044 s <<< FAILURE!
java.lang.AssertionError: assertion failed: Fail to locate the target folder: '/Users/yangjie01/SourceCode/git/spark-maven/sql/connect/server/target'. SPARK_HOME='/Users/yangjie01/SourceCode/git/spark-maven'. Make sure the spark project jars has been built (e.g. using build/sbt package)and the env variable `SPARK_HOME` is set correctly.
  at scala.Predef$.assert(Predef.scala:279)
  at org.apache.spark.sql.connect.test.IntegrationTestUtils$.tryFindJar(IntegrationTestUtils.scala:138)
  at org.apache.spark.sql.connect.test.IntegrationTestUtils$.findJar(IntegrationTestUtils.scala:116)
  at org.apache.spark.sql.connect.test.SparkConnectServerUtils$.sparkConnect$lzycompute(RemoteSparkSession.scala:65)
  at org.apache.spark.sql.connect.test.SparkConnectServerUtils$.sparkConnect(RemoteSparkSession.scala:62)
  at org.apache.spark.sql.connect.test.SparkConnectServerUtils$.start(RemoteSparkSession.scala:135)
  at org.apache.spark.sql.connect.test.SparkConnectServerUtils$.createSparkSession(RemoteSparkSession.scala:181)
  at org.apache.spark.sql.connect.test.SparkConnectServerUtils.createSparkSession(RemoteSparkSession.scala)
  at org.apache.spark.sql.JavaEncoderSuite.setup(JavaEncoderSuite.java:42)
  at java.base/java.lang.reflect.Method.invoke(Method.java:569)
  at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
  Suppressed: java.lang.NullPointerException: Cannot invoke "org.apache.spark.sql.SparkSession.stop()" because "org.apache.spark.sql.JavaEncoderSuite.spark" is null
    at org.apache.spark.sql.JavaEncoderSuite.tearDown(JavaEncoderSuite.java:47)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
    ... 1 more

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   JavaEncoderSuite.setup:42 assertion failed: Fail to locate the target folder: '/Users/yangjie01/SourceCode/git/spark-maven/sql/connect/server/target'. SPARK_HOME='/Users/yangjie01/SourceCode/git/spark-maven'. Make sure the spark project jars has been built (e.g. using build/sbt package)and the env variable `SPARK_HOME` is set correctly.
[INFO]
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
```

To ensure that the above failing scenario can be tested successfully, the current pr makes the following changes:

1. Added `isAssemblyJarsDirExists` in `IntegrationTestUtils` to check if the directory `assembly/target/scala-2.13/jars/` exists.
2. Overrode the `test` function in the `trait RemoteSparkSession` from `AnyFunSuite` to ignore corresponding tests when `isAssemblyJarsDirExists` is `false`. Additionally, `RemoteSparkSession` will only try to start `SparkConnectServer` when `isAssemblyJarsDirExists` is `true`.
3. Made similar changes in `JavaEncoderSuite` as in step 2

### Why are the changes needed?
Make the Maven testing process smoother.

### Does this PR introduce _any_ user-facing change?
No, just for maven test.

### How was this patch tested?
- Pass Git Hub Actions
- Tested the  `connect-client-jvm` module using Maven without `assembly` dir , and the results met expectations.

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #50396 from LuciferYang/SPARK-51603.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: yangjie01 <yangjie01@baidu.com>

Author: LuciferYang

sql/connect/client/jvm/src/test/java/org/apache/spark/sql/JavaEncoderSuite.java

@@ -28,6 +28,7 @@
 import static org.apache.spark.sql.functions.*;
 import static org.apache.spark.sql.RowFactory.create;
 import org.apache.spark.api.java.function.MapFunction;
+import org.apache.spark.sql.connect.test.IntegrationTestUtils;
 import org.apache.spark.sql.connect.test.SparkConnectServerUtils;
 import org.apache.spark.sql.types.StructType;
 
@@ -39,14 +40,18 @@ public class JavaEncoderSuite implements Serializable {
 
   @BeforeAll
   public static void setup() {
+    Assumptions.assumeTrue(IntegrationTestUtils.isAssemblyJarsDirExists(),
+      "Skipping all tests because assembly jars directory does not exist.");
     spark = SparkConnectServerUtils.createSparkSession();
   }
 
   @AfterAll
   public static void tearDown() {
-    spark.stop();
-    spark = null;
-    SparkConnectServerUtils.stop();
+    if (spark != null) {
+      spark.stop();
+      spark = null;
+      SparkConnectServerUtils.stop();
+    }
   }
 
   private static BigDecimal bigDec(long unscaled, int scale) {

sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/test/IntegrationTestUtils.scala

@@ -90,6 +90,11 @@ object IntegrationTestUtils {
     Files.exists(Paths.get(filePath))
   }
 
+  lazy val isAssemblyJarsDirExists: Boolean = {
+    val filePath = s"$sparkHome/assembly/target/$scalaDir/jars/"
+    Files.exists(Paths.get(filePath))
+  }
+
   private[sql] def cleanUpHiveClassesDirIfNeeded(): Unit = {
     def delete(f: File): Unit = {
       if (f.exists()) {

sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/test/RemoteSparkSession.scala

@@ -23,10 +23,12 @@ import java.util.concurrent.TimeUnit
 
 import scala.concurrent.duration.FiniteDuration
 
-import org.scalatest.{BeforeAndAfterAll, Suite}
+import org.scalactic.source.Position
+import org.scalatest.{BeforeAndAfterAll, Suite, Tag}
 import org.scalatest.concurrent.Eventually.eventually
 import org.scalatest.concurrent.Futures.timeout
-import org.scalatest.time.SpanSugar._
+import org.scalatest.funsuite.AnyFunSuite // scalastyle:ignore funsuite
+import org.scalatest.time.SpanSugar._ // scalastyle:ignore
 
 import org.apache.spark.SparkBuildInfo
 import org.apache.spark.sql.connect.SparkSession
@@ -205,14 +207,18 @@ object SparkConnectServerUtils {
   }
 }
 
-trait RemoteSparkSession extends BeforeAndAfterAll { self: Suite =>
+trait RemoteSparkSession
+    extends AnyFunSuite // scalastyle:ignore funsuite
+    with BeforeAndAfterAll { self: Suite =>
   import SparkConnectServerUtils._
   var spark: SparkSession = _
   protected lazy val serverPort: Int = port
 
   override def beforeAll(): Unit = {
     super.beforeAll()
-    spark = createSparkSession()
+    if (IntegrationTestUtils.isAssemblyJarsDirExists) {
+      spark = createSparkSession()
+    }
   }
 
   override def afterAll(): Unit = {
@@ -224,4 +230,13 @@ trait RemoteSparkSession extends BeforeAndAfterAll { self: Suite =>
     spark = null
     super.afterAll()
   }
+
+  override protected def test(testName: String, testTags: Tag*)(testFun: => Any)(implicit
+      pos: Position): Unit = {
+    if (IntegrationTestUtils.isAssemblyJarsDirExists) {
+      super.test(testName, testTags: _*)(testFun)
+    } else {
+      super.ignore(testName, testTags: _*)(testFun)
+    }
+  }
 }