Commit ed20139
[SPARK-49872][FOLLOWUP] Remove Jackson JSON string length limit in KVStoreScalaSerializer
### What changes were proposed in this pull request?
Along the same lines as #52049, apply the same fix to `KVStoreScalaSerializer` as well.
### Why are the changes needed?
The previous fix in #52049 only removed the Jackson JSON string length limit in `JsonProtocol`, which covered Spark event <=> JSON SerDe.
However, the exact same problem exists in `KVStoreScalaSerializer` as well, which is triggered when SHS is configured to use LevelDB/RocksDB KVStores.
Example call chain that triggers the issue:
```
ExecutionPage.render()
→ sqlStore.planGraph(executionId, version)
→ store.read(classOf[SparkPlanGraphWrapper], ...)
→ KVStoreSerializer.deserialize()
→ mapper.readValue() // ← FAILS HERE with 20MB limit
```
### Does this PR introduce _any_ user-facing change?
Yes, users won't hit the Jackson JSON string length limit any more when using SHS.
### How was this patch tested?
Manually tested with a SQL workload that had an intentionally long string for the plan graph.
The fix mechanism itself is tested by previous PRs #49163 and #52049, this is just applying the same fix to SHS as well.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53711 from rednaxelafx/fix-jackson-strlenlimit-kvstore.
Authored-by: Kris Mok <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>1 parent ce6b5a6 commit ed20139
1 file changed
+5
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
76 | 77 | | |
77 | 78 | | |
78 | 79 | | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
79 | 84 | | |
80 | 85 | | |
81 | 86 | | |
| |||
0 commit comments