Skip to content

Commit 7acaa20

Browse files
PavelShilin89PavelShilin89
committed
Add test and documentation for HAVING clause affecting total_found (issue #3428)
1 parent 947d841 commit 7acaa20

File tree

3 files changed

+138
-0
lines changed

3 files changed

+138
-0
lines changed

‎manual/english/Node_info_and_management/SHOW_META.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ The included items are:
1212
* `total_found`:
1313
- The estimated total number of matches for the query in the index. If you need the exact number of matches, use `SELECT COUNT(*)` instead of relying on this value.
1414
- For queries with `GROUP BY`, `total_found` represents the number of groups instead of individual matches.
15+
- When using `HAVING` with `GROUP BY`, `total_found` reflects the number of groups **after** the `HAVING` filter is applied. This enables proper pagination with `HAVING` clauses.
1516
- For [GROUP N BY](../Searching/Grouping.md#Give-me-N-rows) queries, `total_found` still represents the number of groups, regardless of the value of `N`.
1617
* `total_relation`: Indicates whether the `total_found` value is exact or an estimate.
1718
- If Manticore cannot determine the precise `total_found` value, this field will display `total_relation: gte`, meaning the actual number of matches is **Greater Than or Equal** to the reported `total_found`.

‎manual/english/Searching/Grouping.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -767,6 +767,8 @@ SELECT release_year, avg(rental_rate) avg FROM films GROUP BY release_year HAVIN
767767
```
768768
<!-- end -->
769769

770+
**Note:** The `total_found` value in [search query meta info](../Node_info_and_management/SHOW_META.md#SHOW-META) reflects the number of groups that match the `HAVING` condition. This enables proper pagination when using `HAVING` clauses with `GROUP BY`.
771+
770772
<!-- example group7 -->
771773
##### GROUPBY()
772774
There is a function `GROUPBY()` which returns the key of the current group. It's useful in many cases, especially when you [GROUP BY an MVA](../Searching/Grouping.md#Grouping-by-MVA-%28multi-value-attributes%29) or a [JSON value](../Searching/Grouping.md#Grouping-by-a-JSON-node).

‎test/clt-tests/bugs/3428-test-having-total-found.rec‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
––– comment –––
2+
Test for issue #3428: total_found with HAVING clause
3+
https://github.com/manticoresoftware/manticoresearch/issues/3428
4+
5+
Issue: total_found was returning count of all groups, not filtered groups after HAVING.
6+
This broke pagination as total_found didn't reflect actual result count.
7+
8+
Test validates:
9+
1. HAVING filters groups correctly (total_found < all groups)
10+
2. total_found stable across paginated queries
11+
3. Edge cases (empty results, stricter filters)
12+
––– block: ../base/start-searchd-with-buddy –––
13+
––– comment –––
14+
Create table with 10K rows across ~200 categories (uneven distribution for HAVING testing)
15+
––– input –––
16+
manticore-load --quiet --json --init="CREATE TABLE test_having (id bigint, category int, price float, quantity int)" --load="INSERT INTO test_having (id, category, price, quantity) VALUES (<increment>, <int/1/200>, <float/10.0/100.0>, <int/1/50>)" --batch-size=1000 --threads=4 --total=10000 > /dev/null; echo $?
17+
––– output –––
18+
0
19+
––– comment –––
20+
Verify: 10K rows loaded
21+
––– input –––
22+
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM test_having\G"
23+
––– output –––
24+
*************************** 1. row ***************************
25+
count(*): 10000
26+
––– comment –––
27+
Verify: ~200 distinct categories exist
28+
––– input –––
29+
mysql -h0 -P9306 -e "SELECT COUNT(DISTINCT category) as distinct_categories FROM test_having\G"
30+
––– output –––
31+
*************************** 1. row ***************************
32+
distinct_categories: 200
33+
––– comment –––
34+
Verify: data loaded correctly
35+
––– input –––
36+
mysql -h0 -P9306 -e "SELECT * FROM test_having ORDER BY id ASC LIMIT 10000;" > /tmp/test_having_data.txt
37+
––– output –––
38+
––– input –––
39+
md5sum /tmp/test_having_data.txt
40+
––– output –––
41+
e499c54e1ba655eb1e93a51e3e424d5f /tmp/test_having_data.txt
42+
––– comment –––
43+
TEST 1: Get total_found WITH HAVING filter (cnt > 30)
44+
Expected: 199 groups (one category has ≤30 rows)
45+
––– input –––
46+
mysql -h0 -P9306 -e "SELECT category, COUNT(*) as cnt FROM test_having GROUP BY category HAVING cnt > 30 ORDER BY category ASC LIMIT 1\G; SHOW META LIKE 'total_found'\G"
47+
––– output –––
48+
*************************** 1. row ***************************
49+
category: 1
50+
cnt: 62
51+
*************************** 1. row ***************************
52+
Variable_name: total_found
53+
Value: 199
54+
––– comment –––
55+
TEST 2: Get total_found WITHOUT HAVING (baseline - all groups)
56+
Expected: 200 groups (all categories)
57+
––– input –––
58+
mysql -h0 -P9306 -e "SELECT category, COUNT(*) as cnt FROM test_having GROUP BY category ORDER BY category ASC LIMIT 1\G; SHOW META LIKE 'total_found'\G"
59+
––– output –––
60+
*************************** 1. row ***************************
61+
category: 1
62+
cnt: 62
63+
*************************** 1. row ***************************
64+
Variable_name: total_found
65+
Value: 200
66+
––– comment –––
67+
TEST 3: Pagination page 1 (OFFSET 0) - verify total_found
68+
Expected: 199 groups (same as TEST 1)
69+
––– input –––
70+
mysql -h0 -P9306 -e "SELECT category, COUNT(*) as cnt FROM test_having GROUP BY category HAVING cnt > 30 ORDER BY category ASC LIMIT 5 OFFSET 0\G; SHOW META LIKE 'total_found'\G"
71+
––– output –––
72+
*************************** 1. row ***************************
73+
category: 1
74+
cnt: 62
75+
*************************** 2. row ***************************
76+
category: 2
77+
cnt: 51
78+
*************************** 3. row ***************************
79+
category: 3
80+
cnt: 50
81+
*************************** 4. row ***************************
82+
category: 4
83+
cnt: 38
84+
*************************** 5. row ***************************
85+
category: 5
86+
cnt: 53
87+
*************************** 1. row ***************************
88+
Variable_name: total_found
89+
Value: 199
90+
––– comment –––
91+
TEST 4: Pagination page 2 (OFFSET 5) - verify total_found stable
92+
CRITICAL: total_found must be same as page 1 (main issue #3428 complaint)
93+
Expected: 199 groups (same as page 1)
94+
––– input –––
95+
mysql -h0 -P9306 -e "SELECT category, COUNT(*) as cnt FROM test_having GROUP BY category HAVING cnt > 30 ORDER BY category ASC LIMIT 5 OFFSET 5\G; SHOW META LIKE 'total_found'\G"
96+
––– output –––
97+
*************************** 1. row ***************************
98+
category: 6
99+
cnt: 52
100+
*************************** 2. row ***************************
101+
category: 7
102+
cnt: 47
103+
*************************** 3. row ***************************
104+
category: 8
105+
cnt: 59
106+
*************************** 4. row ***************************
107+
category: 9
108+
cnt: 59
109+
*************************** 5. row ***************************
110+
category: 10
111+
cnt: 42
112+
*************************** 1. row ***************************
113+
Variable_name: total_found
114+
Value: 199
115+
––– comment –––
116+
TEST 5: Edge case - HAVING filters everything (cnt > 10000)
117+
Expected: 0 groups (HAVING filters all groups out)
118+
––– input –––
119+
mysql -h0 -P9306 -e "SELECT category, COUNT(*) as cnt FROM test_having GROUP BY category HAVING cnt > 10000\G; SHOW META LIKE 'total_found'\G"
120+
––– output –––
121+
*************************** 1. row ***************************
122+
Variable_name: total_found
123+
Value: 0
124+
––– comment –––
125+
TEST 6: Stricter HAVING filter (cnt > 60)
126+
Expected: fewer groups than cnt > 30 (exactly 15 groups)
127+
––– input –––
128+
mysql -h0 -P9306 -e "SELECT category, COUNT(*) as cnt FROM test_having GROUP BY category HAVING cnt > 60 ORDER BY category ASC LIMIT 1\G; SHOW META LIKE 'total_found'\G"
129+
––– output –––
130+
*************************** 1. row ***************************
131+
category: 1
132+
cnt: 62
133+
*************************** 1. row ***************************
134+
Variable_name: total_found
135+
Value: 15

0 commit comments

Comments
 (0)