Disable FileSystem cache for S3, Azure, and GCS in SSP when using temp credentials (delta-io#6053)

#### Which Delta project/connector is this regarding?

- [x] Spark
- [ ] Standalone
- [ ] Flink
- [ ] Kernel
- [ ] Other (fill in here)

## Description
Set `disable.cache` in the Hadoop configuration built for server-side
planned table reads when injecting temporary credentials from the IRC
response. This ensures each scan uses a fresh FileSystem instance and
avoids AccessDenied when temp credentials expire and a cached FileSystem
is reused.

## How was this patch tested?
Existing server-side planning tests; no new tests (configuration-only
change).

## Does this PR introduce _any_ user-facing changes?
No.

Co-authored-by: Cursor <cursoragent@cursor.com>

Author: murali-db

spark/src/main/scala/org/apache/spark/sql/delta/serverSidePlanning/ServerSidePlannedTable.scala

@@ -402,20 +402,29 @@ class ServerSidePlannedFilePartitionReaderFactory(
   private val hadoopConf = {
     val conf = spark.sessionState.newHadoopConf()
 
-    // Inject temporary credentials from IRC server response
+    // Inject temporary credentials from IRC server response.
+    // Disable FileSystem cache for S3, Azure, and GCS so each scan uses fresh credentials
+    // (avoids AccessDenied when temp creds expire and a cached FS is reused).
+    // Aligns with CredPropsUtil in the Unity Catalog connector.
     credentials.foreach { creds =>
       creds match {
         case S3Credentials(accessKeyId, secretAccessKey, sessionToken) =>
+          conf.set("fs.s3a.path.style.access", "true")
+          conf.set("fs.s3.impl.disable.cache", "true")
+          conf.set("fs.s3a.impl.disable.cache", "true")
           conf.set("fs.s3a.access.key", accessKeyId)
           conf.set("fs.s3a.secret.key", secretAccessKey)
           conf.set("fs.s3a.session.token", sessionToken)
 
         case AzureCredentials(accountName, sasToken, containerName) =>
+          conf.set("fs.abfs.impl.disable.cache", "true")
+          conf.set("fs.abfss.impl.disable.cache", "true")
           // Format: fs.azure.sas.<container>.<account>.dfs.core.windows.net
           val sasKey = s"fs.azure.sas.$containerName.$accountName.dfs.core.windows.net"
           conf.set(sasKey, sasToken)
 
         case GcsCredentials(oauth2Token) =>
+          conf.set("fs.gs.impl.disable.cache", "true")
           conf.set("fs.gs.auth.access.token", oauth2Token)
       }
     }