Skip to content

Add comprehensive tests for force_bigrams option #3909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 26, 2025
Merged

Add comprehensive tests for force_bigrams option #3909

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
14fa292
Add comprehensive tests for force_bigrams in CALL SUGGEST, fuzzy sear…
PavelShilin89 Nov 9, 2025
2906e6d
test: used same keywords with different force_bigrams values for comp…
PavelShilin89 Nov 17, 2025
f6328d1
Added paired tests for force_bigrams with automatic bigram selection,…
PavelShilin89 Nov 19, 2025
cb89b7e
Clarify that Test 5 finds "planet" from "lanet" not because both use …
PavelShilin89 Nov 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 179 additions & 0 deletions test/clt-tests/core/call-autocomplete-force-bigrams-vertical.rec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
––– comment –––
Test for force_bigrams option in CALL AUTOCOMPLETE, CALL SUGGEST, and fuzzy search
Issue: https://github.com/manticoresoftware/manticoresearch/issues/3853

This test verifies that force_bigrams parameter works correctly in three contexts:
1. CALL AUTOCOMPLETE (tests 1-9): autocomplete with bigram-based fuzzy matching
2. CALL SUGGEST (tests 10-14): suggestions with bigram-based fuzzy matching
3. SELECT with OPTION fuzzy=1 (tests 15-19): regular search with fuzzy option

The force_bigrams option helps find transposition errors (e.g., "ipohne" -> "iphone")
in longer words where bigrams are not used by default.
––– block: ../base/start-searchd-with-buddy –––
––– input –––
mysql -P9306 -h0 -e "drop table if exists t"
––– output –––
––– input –––
mysql -P9306 -h0 -e "create table t(f text) min_infix_len='2'"
––– output –––
––– input –––
mysql -P9306 -h0 -e "insert into t values(1,'iphone developer george something planet letter')"
––– output –––
––– comment –––
Test 1: force_bigrams=1 finds transposition errors
––– input –––
mysql -P9306 -h0 -e "call autocomplete('ipohne', 't', 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: iphone
––– comment –––
Test 2: force_bigrams=0 does NOT find transpositions in long words
––– input –––
mysql -P9306 -h0 -e "call autocomplete('ipohne', 't', 0 as force_bigrams)\G"
––– output –––
––– comment –––
Test 3: Long word transposition + force_bigrams
––– input –––
mysql -P9306 -h0 -e "call autocomplete('develepor', 't', 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: developer
––– comment –––
Test 4: Transposition in middle
––– input –––
mysql -P9306 -h0 -e "call autocomplete('geroge', 't', 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: george
––– comment –––
Test 5: Another transposition
––– input –––
mysql -P9306 -h0 -e "call autocomplete('somtehing', 't', 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: something
––– comment –––
Test 6: Short word - bigrams automatic
––– input –––
mysql -P9306 -h0 -e "call autocomplete('lanet', 't', 0 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: planet
––– comment –––
Test 7: Append option
––– input –––
mysql -P9306 -h0 -e "call autocomplete('geor', 't', 1 as append, 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: george
––– comment –––
Test 8: Prepend option
––– input –––
mysql -P9306 -h0 -e "call autocomplete('eorge', 't', 1 as prepend, 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: george
––– comment –––
Test 9: Multiple options
––– input –––
mysql -P9306 -h0 -e "call autocomplete('geor', 't', 1 as fuzziness, 1 as append, 1 as prepend, 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
query: george
––– comment –––
Test 10: CALL SUGGEST with force_bigrams=1 - finds transpositions (SUGGEST is fuzzy by default)
––– input –––
mysql -P9306 -h0 -e "call suggest('ipohne', 't', 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
suggest: iphone
distance: 2
docs: 1
*************************** 2. row ***************************
suggest: planet
distance: 4
docs: 1
––– comment –––
Test 11: CALL SUGGEST with force_bigrams=0 - may not find transpositions in long words
––– input –––
mysql -P9306 -h0 -e "call suggest('ipohne', 't', 0 as force_bigrams)\G"
––– output –––
––– comment –––
Test 12: CALL SUGGEST with force_bigrams=1 - long word transposition
––– input –––
mysql -P9306 -h0 -e "call suggest('develepor', 't', 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
suggest: developer
distance: 2
docs: 1
––– comment –––
Test 13: CALL SUGGEST with force_bigrams=1 - another transposition
––– input –––
mysql -P9306 -h0 -e "call suggest('geroge', 't', 1 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
suggest: george
distance: 2
docs: 1
––– comment –––
Test 14: CALL SUGGEST with force_bigrams=0 - short word (bigrams automatic)
––– input –––
mysql -P9306 -h0 -e "call suggest('lanet', 't', 0 as force_bigrams)\G"
––– output –––
*************************** 1. row ***************************
suggest: planet
distance: 1
docs: 1
*************************** 2. row ***************************
suggest: letter
distance: 4
docs: 1
––– comment –––
Test 15: SELECT with OPTION fuzzy=1, force_bigrams=1 - finds transposition errors
––– input –––
mysql -P9306 -h0 -e "SELECT * FROM t WHERE MATCH('ipohne') OPTION fuzzy=1, force_bigrams=1"
––– output –––
+------+-------------------------------------------------+
| id | f |
+------+-------------------------------------------------+
| 1 | iphone developer george something planet letter |
+------+-------------------------------------------------+
––– comment –––
Test 16: SELECT with OPTION fuzzy=1, force_bigrams=0 - may not find transpositions in long words
––– input –––
mysql -P9306 -h0 -e "SELECT * FROM t WHERE MATCH('ipohne') OPTION fuzzy=1, force_bigrams=0"
––– output –––
––– comment –––
Test 17: SELECT with OPTION fuzzy=1, force_bigrams=1 - long word transposition
––– input –––
mysql -P9306 -h0 -e "SELECT * FROM t WHERE MATCH('develepor') OPTION fuzzy=1, force_bigrams=1"
––– output –––
+------+-------------------------------------------------+
| id | f |
+------+-------------------------------------------------+
| 1 | iphone developer george something planet letter |
+------+-------------------------------------------------+
––– comment –––
Test 18: SELECT with OPTION fuzzy=1, force_bigrams=1 - another transposition
––– input –––
mysql -P9306 -h0 -e "SELECT * FROM t WHERE MATCH('geroge') OPTION fuzzy=1, force_bigrams=1"
––– output –––
+------+-------------------------------------------------+
| id | f |
+------+-------------------------------------------------+
| 1 | iphone developer george something planet letter |
+------+-------------------------------------------------+
––– comment –––
Test 19: SELECT with OPTION fuzzy=1, force_bigrams=0 - short word (bigrams automatic)
––– input –––
mysql -P9306 -h0 -e "SELECT * FROM t WHERE MATCH('lanet') OPTION fuzzy=1, force_bigrams=0"
––– output –––
+------+-------------------------------------------------+
| id | f |
+------+-------------------------------------------------+
| 1 | iphone developer george something planet letter |
+------+-------------------------------------------------+
––– input –––
mysql -P9306 -h0 -e "drop table t"
––– output –––
Loading