Commit d0fbb15
[SPARK-54794][CORE] Suppress verbose
### What changes were proposed in this pull request?
This PR aims to suppress verbose `FsHistoryProvider.checkForLogs` scanning logs.
### Why are the changes needed?
This is a kind of regression at Apache Spark 4.1.0 switched from `logDebug` to `logInfo` accidentally. We had better recover them back to `DEBUG` level because it's excessively verbose currently.
- #52382
```
25/12/21 22:27:53 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356063285
25/12/21 22:28:03 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356073312
25/12/21 22:28:13 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356083338
25/12/21 22:28:23 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356093347
25/12/21 22:28:33 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356103369
25/12/21 22:28:43 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356113391
25/12/21 22:28:53 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356123402
25/12/21 22:29:03 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356133422
25/12/21 22:29:13 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356143443
25/12/21 22:29:23 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356153458
25/12/21 22:29:33 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356163471
25/12/21 22:29:43 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356173491
25/12/21 22:29:53 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356183502
25/12/21 22:30:03 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356193517
25/12/21 22:30:13 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356203529
25/12/21 22:30:23 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356213555
25/12/21 22:30:33 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356223567
25/12/21 22:30:43 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356233585
25/12/21 22:30:53 INFO FsHistoryProvider: Scanning s3a://spark-events with lastScanTime=1766356243599
```
### Does this PR introduce _any_ user-facing change?
No behavior change except log message level.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #53555 from dongjoon-hyun/SPARK-54794.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Kousuke Saruta <[email protected]>FsHistoryProvider.checkForLogs scanning logs1 parent d5b51fe commit d0fbb15
File tree
1 file changed
+1
-1
lines changed- core/src/main/scala/org/apache/spark/deploy/history
1 file changed
+1
-1
lines changedLines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
502 | 502 | | |
503 | 503 | | |
504 | 504 | | |
505 | | - | |
| 505 | + | |
506 | 506 | | |
507 | 507 | | |
508 | 508 | | |
| |||
0 commit comments