Skip to content

DRAFT PoC - Adaptive MinMaxBytesAccumulator with Mode-Sensitive Dense/Sparse Processing and Comprehensive Criterion Benchmarks #18006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 21 commits into from
+4,094 −108
Closed

DRAFT PoC - Adaptive MinMaxBytesAccumulator with Mode-Sensitive Dense/Sparse Processing and Comprehensive Criterion Benchmarks #18006

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
a6e5dec
Implement per-batch scratch tracking for Min/Max bytes accumulator an…
kosiew Oct 6, 2025
8a1b250
minmax-bytes: reintroduce hybrid dense/sparse scratch with heuristic …
kosiew Oct 6, 2025
e8ae111
minmax-bytes: fix dense batch regression and add diagnostic documenta…
kosiew Oct 7, 2025
6e9e171
aggregate-minmax: introduce adaptive workload mode selection and dens…
kosiew Oct 7, 2025
9429b69
minmax-bytes+struct: add sequential dense fast path, deferred marks, and
kosiew Oct 7, 2025
1be6ce1
minmax-bytes: fix overcounting of unique groups in sequential_dense p…
kosiew Oct 9, 2025
afbed2e
functions-aggregate: optimize min/max bytes sequential dense path and…
kosiew Oct 10, 2025
d683cc6
rm working md
kosiew Oct 10, 2025
3238bce
benches: add new benchmarks for extreme duplicates, quadratic growth,…
kosiew Oct 10, 2025
4e3ea35
remove optional tracing dependency and related feature from functions…
kosiew Oct 10, 2025
936dca1
refactor: reorganize imports for clarity in min_max_bytes benchmark
kosiew Oct 10, 2025
cc9f636
fix clippy, taplo errors
kosiew Oct 10, 2025
77cd68b
Migrate staged minima to dense table on limit growth
kosiew Oct 10, 2025
3bbaf4f
Reference externalized test module for min/max bytes
kosiew Oct 10, 2025
31c13e6
Refactor min max test code structure for improved readability and mai…
kosiew Oct 11, 2025
e0c4ca7
Make min_max_bytes module and its structs public for tests and extern…
kosiew Oct 11, 2025
6aef426
revert to before splitting tests.rs
kosiew Oct 11, 2025
d1754e7
Refactor min_max module to directory structure
kosiew Oct 11, 2025
6d0ae78
Register shared tests module and rename inline tests
kosiew Oct 11, 2025
6e0869b
Refactor tests to use as_deref for improved readability and consistency
kosiew Oct 11, 2025
6fbaa98
Add license header to tests.rs for compliance
kosiew Oct 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions datafusion/functions-aggregate/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ datafusion-macros = { workspace = true }
datafusion-physical-expr = { workspace = true }
datafusion-physical-expr-common = { workspace = true }
half = { workspace = true }
hashbrown = { workspace = true }
log = { workspace = true }
paste = "1.0.14"

Expand All @@ -68,3 +69,7 @@ harness = false
[[bench]]
name = "array_agg"
harness = false

[[bench]]
name = "min_max_bytes"
harness = false
Loading