Skip to content

Commit 9355834

Browse files
authored
[VL] Remove dead config spark.gluten.velox.fs.s3a.connect.timeout (#11810)
The config `spark.gluten.velox.fs.s3a.connect.timeout` was originally introduced in GLUTEN-5659 and explicitly read by ConfigExtractor.cc to set the S3 TCP connect timeout (connectTimeoutMs). In GLUTEN-9182, ConfigExtractor.cc was refactored to use a suffix-map approach reading `spark.hadoop.fs.s3a.connection.establish.timeout` for connectTimeoutMs and `spark.hadoop.fs.s3a.connection.timeout` for requestTimeoutMs. The old key was never cleaned up, leaving `AWS_S3_CONNECT_TIMEOUT` defined in VeloxConfig.scala but never referenced, and the key forwarded into the native conf map but never consumed by any C++ code. Remove the config definition, its entries in nativeKeys and getNativeBackendConf defaults, and the documentation row.
1 parent d936025 commit 9355834

File tree

3 files changed

+0
-9
lines changed

3 files changed

+0
-9
lines changed

backends-velox/src/main/scala/org/apache/gluten/config/VeloxConfig.scala

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -551,12 +551,6 @@ object VeloxConfig extends ConfigRegistry {
551551
.stringConf
552552
.createWithDefault("legacy")
553553

554-
val AWS_S3_CONNECT_TIMEOUT =
555-
buildConf("spark.gluten.velox.fs.s3a.connect.timeout")
556-
.doc("Timeout for AWS s3 connection.")
557-
.stringConf
558-
.createWithDefault("200s")
559-
560554
val VELOX_ORC_SCAN_ENABLED =
561555
buildConf("spark.gluten.sql.columnar.backend.velox.orc.scan.enabled")
562556
.doc("Enable velox orc scan. If disabled, vanilla spark orc scan will be used.")

docs/velox-configuration.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,6 @@ nav_order: 16
7878
| spark.gluten.sql.enable.enhancedFeatures | true | Enable some features including iceberg native write and other features. |
7979
| spark.gluten.sql.rewrite.castArrayToString | true | When true, rewrite `cast(array as String)` to `concat('[', array_join(array, ', ', null), ']')` to allow offloading to Velox. |
8080
| spark.gluten.velox.castFromVarcharAddTrimNode | false | If true, will add a trim node which has the same sementic as vanilla Spark to CAST-from-varchar.Otherwise, do nothing. |
81-
| spark.gluten.velox.fs.s3a.connect.timeout | 200s | Timeout for AWS s3 connection. |
8281

8382
## Gluten Velox backend *experimental* configurations
8483

gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -494,7 +494,6 @@ object GlutenConfig extends ConfigRegistry {
494494
SPARK_S3_CONNECTION_MAXIMUM,
495495
SPARK_S3_ENDPOINT_REGION,
496496
SPARK_S3_AWS_IMDS_ENABLED,
497-
"spark.gluten.velox.fs.s3a.connect.timeout",
498497
"spark.gluten.velox.fs.s3a.retry.mode",
499498
"spark.gluten.velox.awsSdkLogLevel",
500499
"spark.gluten.velox.s3UseProxyFromEnv",
@@ -604,7 +603,6 @@ object GlutenConfig extends ConfigRegistry {
604603
(SPARK_S3_USE_INSTANCE_CREDENTIALS, "false"),
605604
(SPARK_S3_RETRY_MAX_ATTEMPTS, "20"),
606605
(SPARK_S3_CONNECTION_MAXIMUM, "15"),
607-
("spark.gluten.velox.fs.s3a.connect.timeout", "200s"),
608606
("spark.gluten.velox.fs.s3a.retry.mode", "legacy"),
609607
(
610608
"spark.gluten.sql.columnar.backend.velox.IOThreads",

0 commit comments

Comments
 (0)