Skip to content

Commit 9b475ed

Browse files
macvincentfacebook-github-bot
authored andcommitted
Remove TODO about Replacing Sorting with Bucketing
Summary: We initially planned to improve the performance of the hard chunking stage by bucketing the streams by size (by most significant bit) instead of sorting them. However, after running benchmarks in D87571945. Sorting always performs better for large stream counts. Differential Revision: D87573831
1 parent 089ba0d commit 9b475ed

File tree

1 file changed

+0
-2
lines changed

1 file changed

+0
-2
lines changed

dwio/nimble/velox/VeloxWriter.cpp

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1144,8 +1144,6 @@ bool VeloxWriter::evalauateFlushPolicy() {
11441144
if (continueChunking) {
11451145
// Relieve memory pressure by chunking small streams.
11461146
// Sort streams for chunking based on raw memory usage.
1147-
// TODO(T240072104): Improve performance by bucketing the streams
1148-
// by size (by most significant bit) instead of sorting them.
11491147
streamIndices.resize(streams.size());
11501148
std::iota(streamIndices.begin(), streamIndices.end(), 0);
11511149
std::sort(

0 commit comments

Comments
 (0)