Commit d7f6020
Refactor Velox Writer to Use New Flush Policy Contract (facebookincubator#242)
Summary:
This should be a no-op since no chunking flush policy is currently being used in Prod. but we make three changes in this dif:
1. `writeChunk` now returns a boolean to indicate whether any stream was successfully chunked
2. The previous raw size of the encoded stripe data in the writer context is now stored in the Writer context
3. We update and pass down the memory stats needed by the new flush policy contract
TODO: We will be introducing two more VeloxWriter changes in the next diffs in this stack to:
1. Support per stream chunking instead of always chunking all eligible streams
2. Support breaking down large stream into multiple smaller chunks
Differential Revision: D815454331 parent 9a7b7cc commit d7f6020
File tree
4 files changed
+428
-109
lines changed- dwio/nimble
- common
- velox
- tests
4 files changed
+428
-109
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
| 77 | + | |
77 | 78 | | |
78 | 79 | | |
79 | 80 | | |
| |||
0 commit comments