Commit 0ecc472
authored
### Rationale for this change
Reduce memory usage required when reading wide, encrypted Parquet files.
### What changes are included in this PR?
Change `SerializedPageReader` so that it doesn't hold a decryption buffer but only allocates one as needed, so it can be freed after pages are decompressed.
### Are these changes tested?
This is only a performance improvement and doesn't change any behaviour so should be covered by existing tests.
The memory improvement has been verified manually (see #46971).
### Are there any user-facing changes?
No
* GitHub Issue: #46971
Authored-by: Adam Reeve <[email protected]>
Signed-off-by: Adam Reeve <[email protected]>
1 parent dadc21f commit 0ecc472
1 file changed
+7
-12
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
226 | 226 | | |
227 | 227 | | |
228 | 228 | | |
229 | | - | |
230 | | - | |
| 229 | + | |
231 | 230 | | |
232 | 231 | | |
233 | 232 | | |
| |||
241 | 240 | | |
242 | 241 | | |
243 | 242 | | |
244 | | - | |
| 243 | + | |
245 | 244 | | |
246 | 245 | | |
247 | 246 | | |
| |||
304 | 303 | | |
305 | 304 | | |
306 | 305 | | |
307 | | - | |
308 | | - | |
309 | 306 | | |
310 | 307 | | |
311 | 308 | | |
| |||
477 | 474 | | |
478 | 475 | | |
479 | 476 | | |
480 | | - | |
481 | | - | |
482 | | - | |
483 | | - | |
484 | | - | |
485 | | - | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
486 | 481 | | |
487 | | - | |
| 482 | + | |
488 | 483 | | |
489 | 484 | | |
490 | 485 | | |
| |||
0 commit comments