Skip to content

feat: Add sort-based shuffle implementation #1389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andygrove merged 18 commits into from
Jan 21, 2026
Merged

feat: Add sort-based shuffle implementation #1389

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
5da659a
feat: Add sort-based shuffle writer (WIP)
andygrove Jan 17, 2026
5e816d4
feat: Integrate sort-based shuffle with planner
andygrove Jan 17, 2026
714858e
feat: Update shuffle reader to support sort-based shuffle and add ben…
andygrove Jan 17, 2026
85653d5
feat: Add executor support for sort-based shuffle and fix index format
andygrove Jan 17, 2026
69ae242
feat: Switch sort shuffle to Arrow IPC File format for random access
andygrove Jan 17, 2026
df378c2
fix: Address clippy warning in shuffle index
andygrove Jan 17, 2026
3e37ade
fix: Address clippy warnings in sort shuffle writer
andygrove Jan 17, 2026
c3b9c01
feat: make query parameter optional in tpch benchmark
andygrove Jan 17, 2026
0607a9e
Merge remote-tracking branch 'apache/main' into sort-based-shuffle
andygrove Jan 17, 2026
1bb9e17
Merge branch 'optional-query-param' into sort-based-shuffle
andygrove Jan 17, 2026
db6b914
feat: add --sort-shuffle flag to tpch benchmark
andygrove Jan 17, 2026
e9f3b28
Merge remote-tracking branch 'apache/main' into sort-based-shuffle
andygrove Jan 17, 2026
d4fcf1a
refactor: return stream instead of Vec<RecordBatch> in sort shuffle r…
andygrove Jan 18, 2026
0445c48
docs: improve sort shuffle module documentation
andygrove Jan 18, 2026
53b5d57
feat: add sort-based shuffle support to flight service
andygrove Jan 18, 2026
a297d47
test: add remote flight read tests for sort-based shuffle
andygrove Jan 18, 2026
4611a45
chore: cargo fmt
andygrove Jan 18, 2026
55fc198
clippy
andygrove Jan 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading