Commit 720ecfe
Dynamically grow hash in linear counting in HLL (elastic#142047) (elastic#142347)
This change dynamically grows the hash inside each linear counting
instance independently to address high memory usage in count_distinct,
especially in cases with a large number of groups (1M+) that each
contain a small number of distinct values (<3000). Instead of eagerly
allocating 16K for each grouping, this dynamic approach serves as an
immediate optimization. The long-term plan, however, is to create a new,
separate HLLState for ES|QL that is more friendly to Block/Vector and
ES|QL aggregations.
Closes elastic#41847
Relates elastic#142333
---------
Co-authored-by: Matt <matthew.alp@elastic.co>
Co-authored-by: Nhat Nguyen <nhat.nguyen@elastic.co>
(cherry picked from commit a0b6bb6)
Co-authored-by: Matt <matthew.alp@elastic.co>1 parent 2347778 commit 720ecfe
File tree
22 files changed
+415
-216
lines changed- docs/changelog
- server/src
- main/java/org/elasticsearch/search/aggregations/metrics
- test/java/org/elasticsearch/search/aggregations/metrics
- x-pack/plugin/esql/compute/src
- main
- generated/org/elasticsearch/compute/aggregation
- java/org/elasticsearch/compute/aggregation
- test/java/org/elasticsearch/compute/aggregation
22 files changed
+415
-216
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
Lines changed: 19 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
99 | 118 | | |
100 | 119 | | |
0 commit comments