Skip to content

Commit 15dcd21

Browse files
tedyuHeartSaVioR
authored andcommitted
[SPARK-50360][SS][FOLLOWUP][MINOR] Make readVersion lazy value
### What changes were proposed in this pull request? This is followup to #48880 `readVersion` is made a `lazy` value. After `input` is closed, its value is not changed to `null` because `input` is only used by `readVersion`. ### Why are the changes needed? `StateStoreChangelogReaderFactory` is able to produce more than one reader. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing test suite. ### Was this patch authored or co-authored using generative AI tooling? No Closes #49091 from tedyu/null-input. Authored-by: Zhihong Yu <[email protected]> Signed-off-by: Jungtaek Lim <[email protected]>
1 parent ef37f9a commit 15dcd21

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreChangelog.scala

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -353,7 +353,7 @@ class StateStoreChangelogReaderFactory(
353353
}
354354
protected val input: DataInputStream = decompressStream(sourceStream)
355355

356-
private def readVersion(): Short = {
356+
private lazy val changeLogVersion: Short = {
357357
try {
358358
val versionStr = input.readUTF()
359359
// Versions in the first line are prefixed with "v", e.g. "v2"
@@ -379,7 +379,7 @@ class StateStoreChangelogReaderFactory(
379379
def constructChangelogReader(): StateStoreChangelogReader = {
380380
var reader: StateStoreChangelogReader = null
381381
try {
382-
reader = readVersion() match {
382+
reader = changeLogVersion match {
383383
case 1 => new StateStoreChangelogReaderV1(fm, fileToRead, compressionCodec)
384384
case 2 => new StateStoreChangelogReaderV2(fm, fileToRead, compressionCodec)
385385
case 3 => new StateStoreChangelogReaderV3(fm, fileToRead, compressionCodec)
@@ -389,6 +389,7 @@ class StateStoreChangelogReaderFactory(
389389
} finally {
390390
if (input != null) {
391391
input.close()
392+
// input is not set to null because it is effectively lazy.
392393
}
393394
}
394395
reader

0 commit comments

Comments
 (0)