Skip to content

Commit 46f9f44

Browse files
WangGuangxinsrowen
authored andcommitted
[SPARK-27202][MINOR][SQL] Update comments to keep according with code
## What changes were proposed in this pull request? Update comments in `InMemoryFileIndex.listLeafFiles` to keep according with code. ## How was this patch tested? existing test cases Closes apache#24146 from WangGuangxin/SPARK-27202. Authored-by: wangguangxin.cn <[email protected]> Signed-off-by: Sean Owen <[email protected]>
1 parent 61d9946 commit 46f9f44

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -308,7 +308,7 @@ object InMemoryFileIndex extends Logging {
308308
// implementations don't actually issue RPC for this method.
309309
//
310310
// - Here we are calling `getFileBlockLocations` in a sequential manner, but it should not
311-
// be a big deal since we always use to `listLeafFilesInParallel` when the number of
311+
// be a big deal since we always use to `bulkListLeafFiles` when the number of
312312
// paths exceeds threshold.
313313
case f =>
314314
// The other constructor of LocatedFileStatus will call FileStatus.getPermission(),

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ abstract class PartitioningAwareFileIndex(
8989
if (partitionSpec().partitionColumns.isEmpty) {
9090
// For each of the root input paths, get the list of files inside them
9191
rootPaths.flatMap { path =>
92-
// Make the path qualified (consistent with listLeafFiles and listLeafFilesInParallel).
92+
// Make the path qualified (consistent with listLeafFiles and bulkListLeafFiles).
9393
val fs = path.getFileSystem(hadoopConf)
9494
val qualifiedPathPre = fs.makeQualified(path)
9595
val qualifiedPath: Path = if (qualifiedPathPre.isRoot && !qualifiedPathPre.isAbsolute) {
@@ -203,7 +203,7 @@ abstract class PartitioningAwareFileIndex(
203203

204204
case None =>
205205
rootPaths.map { path =>
206-
// Make the path qualified (consistent with listLeafFiles and listLeafFilesInParallel).
206+
// Make the path qualified (consistent with listLeafFiles and bulkListLeafFiles).
207207
val qualifiedPath = path.getFileSystem(hadoopConf).makeQualified(path)
208208
if (leafFiles.contains(qualifiedPath)) qualifiedPath.getParent else qualifiedPath }.toSet
209209
}

0 commit comments

Comments
 (0)