feat: optimize infix hash builder with improved radix sort and increased arena size #3981
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sed arena size - Enhanced the infix hash builder by implementing an optimized radix sort for better performance. - Increased the arena size from 1M to 16M buckets to accommodate larger datasets. - Added a new operator for Infix_t to support comparisons. - Improved cache locality by pre-extracting key words during sorting.
Contributor
Linux release test results1 139 tests 1 085 ✅ 15m 53s ⏱️ Results for commit 656f5db. ♻️ This comment has been updated with latest results. |
Contributor
clt❌ CLT tests in Failed tests:🔧 Edit failed tests in UI:
test/clt-tests/buddy-plugins/test-fuzzy-search.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
apt-get update -y > /dev/null; echo $?
––– output –––
OK
––– input –––
apt-get install jq -y > /dev/null; echo $?
––– output –––
OK
––– input –––
php -d memory_limit=2G ./test/clt-tests/scripts/load_names_attr.php --batch-size=100000 --concurrency=1 --docs=1000000 --start-id=1 --min-infix-len=2 > /dev/null; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name ORDER BY id ASC LIMIT 1000000 OPTION max_matches=1000000;" > /tmp/name_data.txt
––– output –––
OK
––– input –––
md5sum /tmp/name_data.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE name;" | grep "min_infix_len='2'" | sed "s/.\(min_infix_len='2'\)./\1/"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM name;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "OPTIMIZE TABLE name OPTION sync=1;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('RICH') ORDER BY id ASC, username ASC OPTION fuzzy=1;"
––– output –––
- +------+---------------------+------+
+ +-------+-----------------+------+
- | id | username | s |
+ | id | username | s |
- +------+---------------------+------+
+ +-------+-----------------+------+
- | 43 | KOURTNEY RICHARDSON | a |
+ | 363 | MAYRA RICO | a |
- | 125 | ELAINA RICHARDSON | a |
+ | 840 | RICO MCFADDEN | a |
- | 226 | RICHARD ZAMARRIPA | a |
+ | 934 | VALERY RICH | a |
- | 363 | MAYRA RICO | a |
+ | 4409 | JACQUELIN RICO | a |
- | 840 | RICO MCFADDEN | a |
+ | 6020 | RICH MAYES | a |
- | 855 | RICHARD ALDRIDGE | a |
+ | 6626 | RICO MCEWEN | a |
- | 934 | VALERY RICH | a |
+ | 9562 | KATHLEEN RICO | a |
- | 948 | REGINE ALDRICH | a |
+ | 11476 | RICO GLADDEN | a |
- | 1017 | HEE TRICE | a |
+ | 13917 | RICO HOLLY | a |
- | 1624 | OLEVIA RICHARDS | a |
+ | 17550 | VIRGINIA RICO | a |
- | 1816 | MAURICE BARCLAY | a |
+ | 17890 | ANGELO RICE | a |
- | 2228 | THERESIA RICHARDS | a |
+ | 18819 | FREDRIC RICO | a |
- | 2670 | JULIANA GRICE | a |
+ | 20555 | JULIA RICH | a |
- | 2918 | GRICELDA WHELAN | a |
+ | 20566 | SHARAN RICE | a |
- | 3012 | RICHARD MCCURDY | a |
+ | 21569 | NICHOLAS RICH | a |
- | 3147 | MERRILEE ALDRICH | a |
+ | 22557 | KYONG RICH | a |
- | 3262 | FONDA RICHEY | a |
+ | 24141 | HORTENSIA RICH | a |
- | 3431 | MELANY RICHARD | a |
+ | 25952 | GRAZYNA RICO | a |
- | 3438 | JOVAN ALDRICH | a |
+ | 27287 | RICO STAUFFER | a |
- | 3517 | BRICE LONGORIA | a |
+ | 27341 | RICO PELLEGRINO | a |
- +------+---------------------+------+
+ +-------+-----------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('EBGRNA') ORDER BY id ASC, username ASC OPTION fuzzy=1;"
––– output –––
- +------+----------------------+------+
- | id | username | s |
- +------+----------------------+------+
- | 20 | BERNARDO DUGGAN | a |
- | 27 | BERNARDINE BREWER | a |
- | 209 | KATLYN SERNA | a |
- | 1292 | LAVERNA CLARK | a |
- | 1414 | BENNY FERNANDEZ | a |
- | 1638 | JULENE BERNAL | a |
- | 2021 | LAVERNA JEFFERSON | a |
- | 2169 | MERNA ORNELAS | a |
- | 2471 | BERNADINE SCANLON | a |
- | 2843 | BERNADINE NAPOLITANO | a |
- | 3177 | FERNANDA FRYER | a |
- | 3199 | FERNANDO FRANCISCO | a |
- | 3455 | FERNANDE MERRITT | a |
- | 3539 | BERNADINE HILLER | a |
- | 3862 | FERNANDO MACDONALD | a |
- | 4022 | JOHNSON HERNANDES | a |
- | 4484 | SYBIL HERNANDES | a |
- | 5159 | MARTINA BERNARD | a |
- | 5793 | BERNA TALBOTT | a |
- | 6506 | FERNANDO FREELAND | a |
- +------+----------------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('ANT') ORDER BY id ASC, username ASC OPTION fuzzy=1;"
––– output –––
- +------+--------------------+------+
- | id | username | s |
- +------+--------------------+------+
- | 6 | KESHIA RANDAZZO | a |
- | 12 | TRESA ESCALANTE | a |
- | 20 | BERNARDO DUGGAN | a |
- | 31 | DEMETRIA IBANEZ | a |
- | 32 | PAMELIA SANDLIN | a |
- | 34 | ANJANETTE MINTON | a |
- | 40 | ANTONIO RIOS | a |
- | 62 | ANDRES NATION | a |
- | 64 | MERI MEDRANO | a |
- | 65 | JANINA FLANIGAN | a |
- | 72 | TALISHA SWAN | a |
- | 75 | JANAY BONNER | a |
- | 80 | DEMETRIA STANFIELD | a |
- | 81 | RALEIGH SANDBERG | a |
- | 82 | KRISTIAN MEYER | a |
- | 83 | JANN HEATH | a |
- | 87 | RANDEE QUINTERO | a |
- | 99 | DEAN GUO | a |
- | 103 | DEAN DEAL | a |
- | 110 | CONSTANCE REES | a |
- +------+--------------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('tony') ORDER BY id ASC, username ASC OPTION fuzzy=1;"
––– output –––
- +-------+------------------+------+
+ +-------+-------------------+------+
- | id | username | s |
+ | id | username | s |
- +-------+------------------+------+
+ +-------+-------------------+------+
- | 688 | SHERIE STORY | a |
+ | 2154 | TONY GREY | a |
- | 1422 | TOBY CARO | a |
+ | 5687 | TONY EASTERLING | a |
- | 1627 | RUPERT STORY | a |
+ | 9619 | TONY WHITTINGTON | a |
- | 2154 | TONY GREY | a |
+ | 13713 | TONY DEBOER | a |
- | 3759 | TOBY GILCHRIST | a |
+ | 15578 | TONY GAY | a |
- | 5687 | TONY EASTERLING | a |
+ | 16078 | TONY CASWELL | a |
- | 5835 | TOBY ROBERSON | a |
+ | 16138 | TONY RABER | a |
- | 6281 | TOBY ARAIZA | a |
+ | 24613 | TONY BRAUN | a |
- | 7311 | ANTONY STOVALL | a |
+ | 25496 | TONY BERGMAN | a |
- | 7475 | TOBY REINHART | a |
+ | 33343 | TONY SOUTHERN | a |
- | 8828 | TONYA CALLAHAN | a |
+ | 40402 | TONY ESPARZA | a |
- | 9251 | TOBY LANGFORD | a |
+ | 42486 | TONY APARICIO | a |
- | 9293 | ZENA STORY | a |
+ | 44087 | TONY DABNEY | a |
- | 9619 | TONY WHITTINGTON | a |
+ | 45395 | TONY MANNING | a |
- | 10167 | LANA STORY | a |
+ | 45967 | TONY HAMMONDS | a |
- | 10341 | ANTONY KAYE | a |
+ | 46732 | TONY SENA | a |
- | 12573 | LATONYA CRABTREE | a |
+ | 49321 | TONY COLEMAN | a |
- | 12876 | TOBY MONTENEGRO | a |
+ | 50358 | TONY HIGGINBOTHAM | a |
- | 13391 | ANTONY CARRILLO | a |
+ | 52342 | TONY WILLARD | a |
- | 13672 | LEZLIE STORY | a |
+ | 53656 | TONY BICKEL | a |
- +-------+------------------+------+
+ +-------+-------------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SMITH') ORDER BY id DESC, username ASC OPTION fuzzy=1;"
––– output –––
- +--------+-----------------+------+
- | id | username | s |
- +--------+-----------------+------+
- | 999737 | KEITH STUMP | a |
- | 999039 | KEITH JAMIESON | a |
- | 996439 | KEITHA CARBAJAL | a |
- | 995770 | KEITH GONZALES | a |
- | 990882 | MINH SANTANA | a |
- | 990643 | KEITH MENDES | a |
- | 989972 | KEITHA WALTERS | a |
- | 989954 | GWENN SMYTH | a |
- | 989444 | KEITH KEATON | a |
- | 988755 | SHERRON SMYTH | a |
- | 988366 | LAVONIA SMYTH | a |
- | 988254 | MAYOLA KEITH | a |
- | 987106 | KEITH FAULK | a |
- | 986580 | DARNELL SMYTH | a |
- | 984012 | KEITH CORONEL | a |
- | 982759 | MORGAN KEITH | a |
- | 982421 | KEITH SHIFFLETT | a |
- | 982223 | TELMA KEITH | a |
- | 982056 | RENITA KEITH | a |
- | 981115 | KATIE SMYTH | a |
- +--------+-----------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SMYTH') ORDER BY id DESC, username ASC OPTION fuzzy=1;"
––– output –––
+--------+----------------+------+
| id | username | s |
+--------+----------------+------+
- | 998829 | EDYTHE VACA | a |
+ | 989954 | GWENN SMYTH | a |
- | 998500 | SETH PHUNG | a |
+ | 988755 | SHERRON SMYTH | a |
- | 996507 | SETH LYMAN | a |
+ | 988366 | LAVONIA SMYTH | a |
- | 995119 | EDYTHE JOHN | a |
+ | 986580 | DARNELL SMYTH | a |
- | 993455 | EDYTH ROSADO | a |
+ | 981115 | KATIE SMYTH | a |
- | 992097 | EDYTH RAGAN | a |
+ | 975841 | ALEIDA SMYTH | a |
- | 989954 | GWENN SMYTH | a |
+ | 974429 | DEMETRIA SMYTH | a |
- | 988823 | SETH WHITTAKER | a |
+ | 973985 | LOGAN SMYTH | a |
- | 988755 | SHERRON SMYTH | a |
+ | 969183 | SCOTTY SMYTH | a |
- | 988366 | LAVONIA SMYTH | a |
+ | 960037 | EDGAR SMYTH | a |
- | 986580 | DARNELL SMYTH | a |
+ | 958857 | MIRTHA SMYTH | a |
- | 986231 | EDYTH BLODGETT | a |
+ | 957495 | SIXTA SMYTH | a |
- | 985595 | SETH BACON | a |
+ | 953233 | LAWRENCE SMYTH | a |
- | 982423 | SETH CHATMAN | a |
+ | 951184 | WILDA SMYTH | a |
- | 981115 | KATIE SMYTH | a |
+ | 944283 | HONG SMYTH | a |
- | 979541 | EDYTHE CLAYTON | a |
+ | 936733 | BEV SMYTH | a |
- | 979093 | SETH BURROUGHS | a |
+ | 935940 | CRISTY SMYTH | a |
- | 978812 | EDYTH GULLEY | a |
+ | 929866 | TEQUILA SMYTH | a |
- | 975994 | SETH PARHAM | a |
+ | 915866 | AMANDA SMYTH | a |
- | 975841 | ALEIDA SMYTH | a |
+ | 908615 | AMBERLY SMYTH | a |
+--------+----------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SMITH') ORDER BY id DESC, username ASC OPTION fuzzy=\"1\";"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('john') ORDER BY id DESC, username ASC OPTION fuzzy=1;"
––– output –––
- +--------+--------------------+------+
+ +--------+-----------------+------+
- | id | username | s |
+ | id | username | s |
- +--------+--------------------+------+
+ +--------+-----------------+------+
- | 999971 | RONNA JOHNSON | a |
+ | 999655 | JOHN WAINWRIGHT | a |
- | 999879 | CHARLEY STJOHN | a |
+ | 999440 | JOHN LEONG | a |
- | 999685 | JOAN KOESTER | a |
+ | 995376 | DOMONIQUE JOHN | a |
- | 999655 | JOHN WAINWRIGHT | a |
+ | 995119 | EDYTHE JOHN | a |
- | 999555 | JON MOSIER | a |
+ | 993439 | ARCHIE JOHN | a |
- | 999440 | JOHN LEONG | a |
+ | 993045 | JOHN HILDRETH | a |
- | 999170 | JONELLE HARDING | a |
+ | 990106 | JOHN CASWELL | a |
- | 999075 | JOHNIE DURAN | a |
+ | 988428 | HELAINE JOHN | a |
- | 999020 | JONAS HASAN | a |
+ | 987946 | JOHN USHER | a |
- | 998802 | RENETTA LITTLEJOHN | a |
+ | 986706 | JOHN BOHN | a |
- | 998721 | REY JOHNSTON | a |
+ | 985882 | JOHN BREWSTER | a |
- | 998368 | JONIE MILTON | a |
+ | 985305 | TAUNYA JOHN | a |
- | 998367 | LEENA JOHNSTON | a |
+ | 982110 | CELINA JOHN | a |
- | 998128 | SHARILYN JOHNSTON | a |
+ | 980235 | JOHN HARDMAN | a |
- | 997788 | JOANNIE HENSON | a |
+ | 977317 | JOHN JORGENSEN | a |
- | 997670 | DENNY JONES | a |
+ | 977053 | JOHN ESCOBEDO | a |
- | 997623 | JOHNETTA DELATORRE | a |
+ | 975483 | WELDON JOHN | a |
- | 997577 | JONA CORRAL | a |
+ | 974397 | JOHN PATE | a |
- | 997563 | JONE FOLSOM | a |
+ | 973982 | KIRA JOHN | a |
- | 997512 | JOAN CALKINS | a |
+ | 973504 | JOHN TERAN | a |
- +--------+--------------------+------+
+ +--------+-----------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('jane') AND id BETWEEN 10000 AND 20000 ORDER BY username ASC OPTION fuzzy=1;"
––– output –––
- +-------+----------------------+------+
+ +-------+---------------+------+
- | id | username | s |
+ | id | username | s |
- +-------+----------------------+------+
+ +-------+---------------+------+
- | 10119 | JANIECE GIBSON | a |
+ | 12608 | JAN DUPONT | a |
- | 10159 | JANNA MIXON | a |
+ | 13421 | JAN MARR | a |
- | 10200 | ELANE LOPES | a |
+ | 14313 | JAN CHAN | a |
- | 10205 | JANETTA ZHOU | a |
+ | 15190 | JAN BARRON | a |
- | 10381 | EULALIA LANE | a |
+ | 17623 | JAN MCWHORTER | a |
- | 10465 | JANIE PAXTON | a |
+ +-------+---------------+------+
- | 10492 | JANINA JOE | a |
- | 10507 | JANNA EDDY | a |
- | 10635 | JANESSA GOODIN | a |
- | 10637 | JACQUELYNN JANKOWSKI | a |
- | 10824 | ALEJANDRINA LAUGHLIN | a |
- | 10831 | CLYDE JANES | a |
- | 10926 | JANNET GOODEN | a |
- | 10978 | JANAE KELSO | a |
- | 11064 | JANETH BOWIE | a |
- | 11266 | MARIELLE ALEJANDRO | a |
- | 11342 | JANAE DELAGARZA | a |
- | 11875 | TYLER LUJAN | a |
- | 11925 | DINAH JANES | a |
- | 11997 | JANETH BROWNING | a |
- +-------+----------------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('EBGRNA') AND id BETWEEN 1000 AND 2000 ORDER BY username ASC OPTION fuzzy=1;"
––– output –––
- +------+-----------------+------+
- | id | username | s |
- +------+-----------------+------+
- | 1292 | LAVERNA CLARK | a |
- | 1414 | BENNY FERNANDEZ | a |
- | 1638 | JULENE BERNAL | a |
- +------+-----------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SARA') ORDER BY id ASC, username ASC OPTION fuzzy=1;"
––– output –––
- +------+----------------------+------+
+ +-------+----------------+------+
- | id | username | s |
+ | id | username | s |
- +------+----------------------+------+
+ +-------+----------------+------+
- | 73 | CLARA NGUYEN | a |
+ | 2305 | SARA AVILA | a |
- | 1285 | CLARA CARTAGENA | a |
+ | 6165 | SARA HALLMAN | a |
- | 1515 | KLARA ALARCON | a |
+ | 8822 | SARA WEINBERG | a |
- | 1709 | FREDDA LARA | a |
+ | 10826 | SARA TELLEZ | a |
- | 2217 | LORIANN DELROSARIO | a |
+ | 11443 | SARA EDEN | a |
- | 2305 | SARA AVILA | a |
+ | 18085 | SARA SIMS | a |
- | 2631 | ROSARIA DELONG | a |
+ | 22346 | SARA STEINBERG | a |
- | 3245 | LARA MASTERSON | a |
+ | 29990 | SARA SUMPTER | a |
- | 3267 | LARA MOREAU | a |
+ | 30605 | SARA DALEY | a |
- | 3553 | SARAN CHANG | a |
+ | 37233 | SARA ALBERTSON | a |
- | 3880 | SARAN HOOK | a |
+ | 38382 | SARA CRAMER | a |
- | 3980 | GEORGETTA DELROSARIO | a |
+ | 40067 | SARA QUINTANA | a |
- | 4517 | SARAI EASLEY | a |
+ | 51593 | SARA RIGGINS | a |
- | 5708 | ASA ROSARIO | a |
+ | 53815 | SARA WAGNER | a |
- | 5731 | SARITA ANDERS | a |
+ | 65455 | SARA PAPPAS | a |
- | 6100 | CHANTELL SARABIA | a |
+ | 81221 | SARA BRILL | a |
- | 6165 | SARA HALLMAN | a |
+ | 90393 | SARA CURRAN | a |
- | 6480 | SARAH NEILL | a |
+ | 91455 | SARA BARRAGAN | a |
- | 6488 | SARAN PARTIN | a |
+ | 92815 | SARA NATHAN | a |
- | 6802 | KLARA ROLLER | a |
+ | 95651 | SARA AGUILAR | a |
- +------+----------------------+------+
+ +-------+----------------+------+
––– input –––
mysql -h0 -P9306 -e "SELECT username FROM name WHERE MATCH('SMITH') OPTION cutoff=0, ranker=expr('sum((4*lcs+2*(min_hit_pos=1)+exact_hit)*user_weight)*1000+bm25'), field_weights=(username=1), fuzzy=1;"
––– output –––
- +----------------+
- | username |
- +----------------+
- | MINH FENNELL |
- | MINH MAZUR |
- | MINH VILLEGAS |
- | MINH TILLER |
- | MINH VOGT |
- | MINH SHAW |
- | MINH STEADMAN |
- | MINH PEAK |
- | MINH ROOT |
- | MINH PATTERSON |
- | MINH BURDETTE |
- | MINH ABNEY |
- | MINH DOLAN |
- | MINH RUDD |
- | MINH TOLBERT |
- | MINH CRANDALL |
- | MINH JAIME |
- | MINH ISAACSON |
- | MINH BONILLA |
- | MINH WAHL |
- +----------------+
––– input –––
mysql -h0 -P9306 -e "SELECT username FROM name WHERE MATCH('SMITH') OPTION cutoff=0, field_weights=(username=1), fuzzy=1;"
––– output –––
- +-----------------+
- | username |
- +-----------------+
- | SYDNEY SMYTH |
- | CHERILYN SMYTH |
- | SATURNINA SMYTH |
- | LYNN SMYTH |
- | STEVEN SMYTH |
- | JASMIN SMYTH |
- | BERYL SMYTH |
- | JACK SMYTH |
- | CAROLEE SMYTH |
- | CARON SMYTH |
- | ALFREDO SMYTH |
- | ALTA SMYTH |
- | VIKI SMYTH |
- | CHANA SMYTH |
- | HYUN SMYTH |
- | ROGER SMYTH |
- | LINDSEY SMYTH |
- | WILFORD SMYTH |
- | ARLINE SMYTH |
- | APRYL SMYTH |
- +-----------------+
––– input –––
mysql -h0 -P9306 -e "SELECT username FROM name WHERE MATCH('SMITH') OPTION cutoff=0, ranker=expr('sum((4*lcs+2*(min_hit_pos=1)+exact_hit)*user_weight)*1000+bm25'), fuzzy=1;"
––– output –––
- +----------------+
- | username |
- +----------------+
- | MINH FENNELL |
- | MINH MAZUR |
- | MINH VILLEGAS |
- | MINH TILLER |
- | MINH VOGT |
- | MINH SHAW |
- | MINH STEADMAN |
- | MINH PEAK |
- | MINH ROOT |
- | MINH PATTERSON |
- | MINH BURDETTE |
- | MINH ABNEY |
- | MINH DOLAN |
- | MINH RUDD |
- | MINH TOLBERT |
- | MINH CRANDALL |
- | MINH JAIME |
- | MINH ISAACSON |
- | MINH BONILLA |
- | MINH WAHL |
- +----------------+
––– input –––
mysql -h0 -P9306 -e "SELECT username FROM name WHERE MATCH('SMITH') OPTION cutoff=0, fuzzy=1;"
––– output –––
- +-----------------+
- | username |
- +-----------------+
- | SYDNEY SMYTH |
- | CHERILYN SMYTH |
- | SATURNINA SMYTH |
- | LYNN SMYTH |
- | STEVEN SMYTH |
- | JASMIN SMYTH |
- | BERYL SMYTH |
- | JACK SMYTH |
- | CAROLEE SMYTH |
- | CARON SMYTH |
- | ALFREDO SMYTH |
- | ALTA SMYTH |
- | VIKI SMYTH |
- | CHANA SMYTH |
- | HYUN SMYTH |
- | ROGER SMYTH |
- | LINDSEY SMYTH |
- | WILFORD SMYTH |
- | ARLINE SMYTH |
- | APRYL SMYTH |
- +-----------------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SMITH') ORDER BY id ASC OPTION fuzzy=0;" > /tmp/result_fuzzy0.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SMITH') ORDER BY id ASC;" > /tmp/result_no_fuzzy.txt
––– output –––
OK
––– input –––
diff /tmp/result_fuzzy0.txt /tmp/result_no_fuzzy.txt; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 1 AS fuzziness);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 'us,ru' AS layouts);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 1 AS append);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 0 AS prepend);" | tail -n +4 | sort
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 1 AS prepend, 1 AS append);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 1 AS fuzziness, '' AS layouts, 1 AS append, 1 AS prepend);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 2 AS fuzziness, 'us,ru' AS layouts, 1 AS append, 1 AS prepend, 2 AS expansion_len);" | tail -n +4 | sort
––– output –––
- +-------+
- | joe |
- | joey |
- | john |
- | jon |
- | jose |
- | joy |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 1 AS append);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 1 AS prepend);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 1 AS append, 1 AS prepend);" | tail -n +4 | sort
––– output –––
- +--------+
- | joe |
- | joey |
- | john |
- | jon |
- | jordan |
- | jose |
- | joseph |
- | joy |
- | joyce |
- | major |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 'us' AS layouts);" | tail -n +4 | sort
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('jo', 'name', 'ru' AS layouts);" | tail -n +4 | sort
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('al', 'name', 0 AS fuzziness);" | tail -n +4 | sort
––– output –––
- +-----------+
- | albert |
- | alexander |
- | ali |
- | dale |
- | gale |
- | kendall |
- | marshall |
- | palma |
- | valentine |
- | walter |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('mi', 'name', 0 AS append, 1 AS prepend);" | tail -n +4 | sort
––– output –––
- +--------+
- | cami |
- | darci |
- | joan |
- | jody |
- | joe |
- | joel |
- | joey |
- | john |
- | johnie |
- | johnny |
- | joi |
- | jon |
- | jonas |
- | jordan |
- | jorge |
- | jose |
- | joseph |
- | joshi |
- | joshua |
- | joy |
- | joyce |
- | kaci |
- | kami |
- | kimi |
- | laci |
- | luci |
- | marci |
- | mi |
- | nanci |
- | naomi |
- | neomi |
- | nohemi |
- | ricci |
- | staci |
- | tami |
- | tammi |
- | tobias |
- | todd |
- | toi |
- | tom |
- | tomas |
- | tomi |
- | toney |
- | traci |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('ch', 'name', 1 AS fuzziness, 3 AS expansion_len);" | tail -n +4 | sort
––– output –––
- +-------+
- | chang |
- | chong |
- | chung |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('зф', 'name', 'us,ru' AS layouts);" | tail -n +4 | sort
––– output –––
- +----------+
- | page |
- | paige |
- | palma |
- | paris |
- | parker |
- | pat |
- | patricia |
- | patrick |
- | paul |
- | paz |
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('зф', 'name', 'us' AS layouts);" | tail -n +4 | sort
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('gr', 'name', 1 AS fuzziness, 'ru' AS layouts, 1 AS append, 1 AS prepend);" | tail -n +4 | sort
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DROP TABLE IF EXISTS tbl1; CREATE TABLE tbl1 (title TEXT) min_infix_len='2';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM \`tbl1\` WHERE MATCH('abc') OPTION fuzzy = 1;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "drop table if exists t; create table t(f text) min_infix_len='2'; insert into t values(1, 'something'), (2, 'some thing'); select * from t where match('somethin') option fuzzy=0;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select * from t where match('somethin') option fuzzy=0;"
––– output –––
OK
––– input –––
mysql -P9306 -h0 -e "drop table if exists t; create table t(f text) min_infix_len='2'; insert into t values(1, 'something'), (2, 'some thing'); call autocomplete('some', 't');"
––– output –––
OK
––– input –––
mysql -P9306 -h0 -e "drop table if exists t; create table t(f text) min_infix_len='2'; insert into t values(1, 'something'), (2, 'some thing'); call autocomplete('some', 't');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "drop table if exists t; create table t(a int);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select * from t option fuzzy=1;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "drop table if exists comment_28798446; create table comment_28798446(id int, content text) min_infix_len='3';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "insert into comment_28798446 values (1, 'business Manticore interaction'), (2, 'buisness Manticore interation'), (3, 'other text content');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM comment_28798446 WHERE MATCH('buisness Manticore interation') OPTION fuzzy = 1, layouts = '';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM comment_28798446 WHERE MATCH('business Manticore interaction') OPTION fuzzy = 1, layouts = '';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM comment_28798446 WHERE MATCH('Manticore') OPTION fuzzy = 1, layouts = '';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('bus', 'comment_28798446', 'ru,us,ua' AS layouts);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('bus', 'comment_28798446', '' AS layouts);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "drop table if exists t; create table t(f text) min_infix_len='2'; insert into t values(1, 'something'), (2, 'some thing'); select * from t where match('somethin') option fuzzy=0;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select * from t where match('somethin') option fuzzy=0;"
––– output –––
OK
––– input –––
mysql -P9306 -h0 -e "drop table if exists t; create table t(f text) min_infix_len='2'; insert into t values(1, 'something'), (2, 'some thing'); call autocomplete('some', 't');"
––– output –––
OK
––– input –––
mysql -P9306 -h0 -e "drop table if exists t; create table t(f text) min_infix_len='2'; insert into t values(1, 'something'), (2, 'some thing'); call autocomplete('some', 't');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "drop table if exists t; create table t(a int);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select * from t option fuzzy=1;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "drop table if exists comment_28798446; create table comment_28798446(id int, content text) min_infix_len='3';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "insert into comment_28798446 values (1, 'business Manticore interaction'), (2, 'buisness Manticore interation'), (3, 'other text content');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM comment_28798446 WHERE MATCH('buisness Manticore interation') OPTION fuzzy = 1, layouts = '';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM comment_28798446 WHERE MATCH('business Manticore interaction') OPTION fuzzy = 1, layouts = '';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM comment_28798446 WHERE MATCH('Manticore') OPTION fuzzy = 1, layouts = '';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('bus', 'comment_28798446', 'ru,us,ua' AS layouts);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('bus', 'comment_28798446', '' AS layouts);"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DROP TABLE IF EXISTS t; DEBUG SLEEP 2; CREATE TABLE t(f TEXT) min_infix_len='2';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO t VALUES (1, 'abcdef'), (2, 'abcd efgh');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM t WHERE MATCH('abcdef'); SHOW META;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "insert into t values(3, 'aa defghi xxx');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "select * from t where match('aa def ghi xxx') option fuzzy=1, layouts='';"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM t WHERE MATCH('abcef') OPTION fuzzy=1; SHOW META;"
––– output –––
OK
––– input –––
grep -A3 "SELECT \* FROM t WHERE MATCH('abcdef')" /var/log/manticore/query.log
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM t WHERE MATCH('nonexistent'); SHOW META;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM t WHERE MATCH('abcdef') OPTION fuzzy=1;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM t WHERE MATCH('\$#@\!') OPTION fuzzy=1;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DROP TABLE IF EXISTS idx1; CREATE TABLE idx1(value TEXT) min_infix_len='2';"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO idx1(value) VALUES ('340');"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('34', 'idx1');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('3', 'idx1');"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('34', 'idx1', 1 AS fuzziness);"
––– output –––
OK
––– input –––
echo -e "searchd {\nlisten = 9306:mysql\nlisten = 9312\nlisten = 9308:http\nlog = /tmp/searchd.log\nquery_log = /tmp/query.log\npid_file = /tmp/searchd.pid\n}\n\nsource src_idx {\ntype = tsvpipe\ntsvpipe_command = cat /tmp/data.tsv\ntsvpipe_field_string = f\ntsvpipe_attr_bigint = id\n}\n\nindex idx_plain {\ntype = plain\nsource = src_idx\npath = /tmp/idx_plain\nmin_infix_len = 2\n}" > /tmp/manticore_idx.conf; echo $?
––– output –––
OK
––– input –––
echo -e "1\tabcdef\n2\tabcd efgh\n3\trandom text" > /tmp/data.tsv; echo $?
––– output –––
OK
––– input –––
mkdir -p /tmp/idx_plain && chmod 777 /tmp/idx_plain; echo $?
––– output –––
OK
––– input –––
indexer --all --config /tmp/manticore_idx.conf > /dev/null 2>&1; echo $?
––– output –––
OK
––– input –––
rm -f /tmp/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; (stdbuf -oL searchd -c /tmp/manticore_idx.conf 2>&1 | tee /tmp/searchd.log | grep -i precach & disown); sleep 0.5; if timeout 10 grep -qm1 '\[BUDDY\] started' /tmp/searchd.log; then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /tmp/searchd.log; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW TABLES;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW TABLE idx_plain SETTINGS;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM idx_plain WHERE MATCH('abcdef') OPTION fuzzy=1;"
––– output –––
- +------+-----------+
+ +------+--------+
- | id | f |
+ | id | f |
- +------+-----------+
+ +------+--------+
- | 1 | abcdef |
+ | 1 | abcdef |
- | 2 | abcd efgh |
+ +------+--------+
- +------+-----------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM idx_plain WHERE MATCH('abcef') OPTION fuzzy=1;"
––– output –––
- +------+-----------+
- | id | f |
- +------+-----------+
- | 1 | abcdef |
- | 2 | abcd efgh |
- +------+-----------+
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM idx_plain WHERE MATCH('nonexistent') OPTION fuzzy=1;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('abc', 'idx_plain');"
––– output –––
- +--------+
- | query |
- +--------+
- | abcd |
- | abcdef |
- +--------+
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('ran', 'idx_plain');"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('zzz', 'idx_plain');"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM idx_plain WHERE MATCH('abcdef') OPTION fuzzy=0;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM idx_plain WHERE MATCH('abcd efgh') OPTION fuzzy=0;"
––– output –––
OKtest/clt-tests/buddy-plugins/test-fuzzy-autocomplete-http.rec––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
apt-get update -y > /dev/null; echo $?
––– output –––
OK
––– input –––
apt-get install jq -y > /dev/null; echo $?
––– output –––
OK
––– input –––
php -d memory_limit=2G ./test/clt-tests/scripts/load_names_attr.php --batch-size=100000 --concurrency=1 --docs=1000000 --start-id=1 --min-infix-len=2 > /dev/null; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM name ORDER BY id ASC LIMIT 1000000 OPTION max_matches=1000000;" > /tmp/name_data.txt
––– output –––
OK
––– input –––
md5sum /tmp/name_data.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SHOW CREATE TABLE name;" | grep "min_infix_len='2'" | sed "s/.\(min_infix_len='2'\)./\1/"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM name;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "OPTIMIZE TABLE name OPTION sync=1;"
––– output –––
OK
––– input –––
curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"EBGRNA"}}]}},"options":{"fuzzy":true}}' | jq -r '.hits.hits[]._source.username'
––– output –––
- BERNA PFEIFFER
- BERNA SAGE
- BERNA PANG
- BERNA STOLTZFUS
- BERNA HALVERSON
- BERNA PLASCENCIA
- BERNA CALLAWAY
- BERNA STONER
- BERNA MARSH
- BERNA LEI
- BERNA SOUSA
- BERNA LIAO
- BERNA TROTTER
- BERNA VALDOVINOS
- BERNA VALLE
- BERNA ELKINS
- BERNA HURD
- BERNA DUARTE
- BERNA HAUGEN
- BERNA LISTER
––– input –––
curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"ANT"}}]}},"options":{"fuzzy":true}}' | jq -r '.hits.hits[]._source.username'
––– output –––
- SHANE GANT
- DANILLE GANT
- BLANCH GANT
- TRISTAN GANT
- NATHANAEL GANT
- JANESSA GANT
- ANDREW GANT
- KORTNEY GANT
- SABINE GANT
- IRA GANT
- SHAWNTA GANT
- TRACEY GANT
- JIM GANT
- GERRY GANT
- ELROY GANT
- ERIN GANT
- LEONARDA GANT
- JARED GANT
- RODNEY GANT
- ROCKY GANT
––– input –––
curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"tony"}}]}},"options":{"fuzzy":true}}' | jq -M
––– output –––
{
"took": %{NUMBER},
"timed_out": false,
"hits": {
- "total": 1813,
+ "total": 353,
"total_relation": "eq",
"hits": [
{
- "_id": 800363,
+ "_id": 801407,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY BEAULIEU",
+ "username": "TONY KERSEY",
"s": "a"
}
},
{
- "_id": 807095,
+ "_id": 802954,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY HOFER",
+ "username": "TONY JACOBSON",
"s": "a"
}
},
{
- "_id": 807096,
+ "_id": 804215,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY DEMPSEY",
+ "username": "TONY WILLIS",
"s": "a"
}
},
{
- "_id": 808820,
+ "_id": 807893,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY RAYMOND",
+ "username": "TONY TREMBLAY",
"s": "a"
}
},
{
- "_id": 808863,
+ "_id": 809344,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY BATEMAN",
+ "username": "TONY CONNORS",
"s": "a"
}
},
{
- "_id": 813761,
+ "_id": 810439,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY VANWINKLE",
+ "username": "TONY GODINEZ",
"s": "a"
}
},
{
- "_id": 818229,
+ "_id": 811284,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY BROCK",
+ "username": "TONY ATWOOD",
"s": "a"
}
},
{
- "_id": 825472,
+ "_id": 822885,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY LINDQUIST",
+ "username": "TONY ACKERMAN",
"s": "a"
}
},
{
- "_id": 826251,
+ "_id": 831416,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY KAUFFMAN",
+ "username": "TONY HERRINGTON",
"s": "a"
}
},
{
- "_id": 827886,
+ "_id": 833357,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY PITMAN",
+ "username": "TONY HYLAND",
"s": "a"
}
},
{
- "_id": 832558,
+ "_id": 833493,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY KELLEY",
+ "username": "TONY SAUNDERS",
"s": "a"
}
},
{
- "_id": 833689,
+ "_id": 834620,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY HANDY",
+ "username": "TONY CROCKETT",
"s": "a"
}
},
{
- "_id": 834136,
+ "_id": 837319,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY HOPPE",
+ "username": "TONY WEINER",
"s": "a"
}
},
{
- "_id": 836413,
+ "_id": 838630,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY STOVALL",
+ "username": "TONY IBANEZ",
"s": "a"
}
},
{
- "_id": 839075,
+ "_id": 840247,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY JEROME",
+ "username": "TONY HAMBY",
"s": "a"
}
},
{
- "_id": 839512,
+ "_id": 842966,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY LEONARD",
+ "username": "TONY MURDOCK",
"s": "a"
}
},
{
- "_id": 841923,
+ "_id": 846296,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY PARNELL",
+ "username": "TONY BUSH",
"s": "a"
}
},
{
- "_id": 842450,
+ "_id": 852863,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY LOWE",
+ "username": "TONY DELAGARZA",
"s": "a"
}
},
{
- "_id": 843217,
+ "_id": 853630,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY ROBISON",
+ "username": "TONY KRIEGER",
"s": "a"
}
},
{
- "_id": 846291,
+ "_id": 857623,
- "_score": 1547,
+ "_score": 1644,
"_source": {
- "username": "TOBY BITTNER",
+ "username": "TONY GAYTAN",
"s": "a"
}
}
]
}
}
––– input –––
curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"SMITH"}}]}}, "sort":[{"id":"desc"},{"username":"asc"}], "options":{"fuzzy":true}}' | jq -M
––– output –––
{
"took": %{NUMBER},
"timed_out": false,
"hits": {
- "total": 1304,
+ "total": 0,
"total_relation": "eq",
- "hits": [
+ "hits": []
- {
+ }
- "_id": 999737,
+ }
- "_score": 1,
- "_source": {
- "username": "KEITH STUMP",
- "s": "a"
- }
- },
- {
- "_id": 999039,
- "_score": 1,
- "_source": {
- "username": "KEITH JAMIESON",
- "s": "a"
- }
- },
- {
- "_id": 996439,
- "_score": 1,
- "_source": {
- "username": "KEITHA CARBAJAL",
- "s": "a"
- }
- },
- {
- "_id": 995770,
- "_score": 1,
- "_source": {
- "username": "KEITH GONZALES",
- "s": "a"
- }
- },
- {
- "_id": 990882,
- "_score": 1,
- "_source": {
- "username": "MINH SANTANA",
- "s": "a"
- }
- },
- {
- "_id": 990643,
- "_score": 1,
- "_source": {
- "username": "KEITH MENDES",
- "s": "a"
- }
- },
- {
- "_id": 989972,
- "_score": 1,
- "_source": {
- "username": "KEITHA WALTERS",
- "s": "a"
- }
- },
- {
- "_id": 989954,
- "_score": 1,
- "_source": {
- "username": "GWENN SMYTH",
- "s": "a"
- }
- },
- {
- "_id": 989444,
- "_score": 1,
- "_source": {
- "username": "KEITH KEATON",
- "s": "a"
- }
- },
- {
- "_id": 988755,
- "_score": 1,
- "_source": {
- "username": "SHERRON SMYTH",
- "s": "a"
- }
- },
- {
- "_id": 988366,
- "_score": 1,
- "_source": {
- "username": "LAVONIA SMYTH",
- "s": "a"
- }
- },
- {
- "_id": 988254,
- "_score": 1,
- "_source": {
- "username": "MAYOLA KEITH",
- "s": "a"
- }
- },
- {
- "_id": 987106,
- "_score": 1,
- "_source": {
- "username": "KEITH FAULK",
- "s": "a"
- }
- },
- {
- "_id": 986580,
- "_score": 1,
- "_source": {
- "username": "DARNELL SMYTH",
- "s": "a"
- }
- },
- {
- "_id": 984012,
- "_score": 1,
- "_source": {
- "username": "KEITH CORONEL",
- "s": "a"
- }
- },
- {
- "_id": 982759,
- "_score": 1,
- "_source": {
- "username": "MORGAN KEITH",
- "s": "a"
- }
- },
- {
- "_id": 982421,
- "_score": 1,
- "_source": {
- "username": "KEITH SHIFFLETT",
- "s": "a"
- }
- },
- {
- "_id": 982223,
- "_score": 1,
- "_source": {
- "username": "TELMA KEITH",
- "s": "a"
- }
- },
- {
- "_id": 982056,
- "_score": 1,
- "_source": {
- "username": "RENITA KEITH",
- "s": "a"
- }
- },
- {
- "_id": 981115,
- "_score": 1,
- "_source": {
- "username": "KATIE SMYTH",
- "s": "a"
- }
- }
- ]
- }
- }
––– input –––
curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"SMYTH"}}]}}, "sort":[{"id":"desc"},{"username":"asc"}], "options":{"fuzzy":true}}' | jq -M
––– output –––
{
"took": %{NUMBER},
"timed_out": false,
"hits": {
- "total": 773,
+ "total": 216,
"total_relation": "eq",
"hits": [
{
- "_id": 998829,
+ "_id": 989954,
"_score": 1,
"_source": {
- "username": "EDYTHE VACA",
+ "username": "GWENN SMYTH",
"s": "a"
}
},
{
- "_id": 998500,
+ "_id": 988755,
"_score": 1,
"_source": {
- "username": "SETH PHUNG",
+ "username": "SHERRON SMYTH",
"s": "a"
}
},
{
- "_id": 996507,
+ "_id": 988366,
"_score": 1,
"_source": {
- "username": "SETH LYMAN",
+ "username": "LAVONIA SMYTH",
"s": "a"
}
},
{
- "_id": 995119,
+ "_id": 986580,
"_score": 1,
"_source": {
- "username": "EDYTHE JOHN",
+ "username": "DARNELL SMYTH",
"s": "a"
}
},
{
- "_id": 993455,
+ "_id": 981115,
"_score": 1,
"_source": {
- "username": "EDYTH ROSADO",
+ "username": "KATIE SMYTH",
"s": "a"
}
},
{
- "_id": 992097,
+ "_id": 975841,
"_score": 1,
"_source": {
- "username": "EDYTH RAGAN",
+ "username": "ALEIDA SMYTH",
"s": "a"
}
},
{
- "_id": 989954,
+ "_id": 974429,
"_score": 1,
"_source": {
- "username": "GWENN SMYTH",
+ "username": "DEMETRIA SMYTH",
"s": "a"
}
},
{
- "_id": 988823,
+ "_id": 973985,
"_score": 1,
"_source": {
- "username": "SETH WHITTAKER",
+ "username": "LOGAN SMYTH",
"s": "a"
}
},
{
- "_id": 988755,
+ "_id": 969183,
"_score": 1,
"_source": {
- "username": "SHERRON SMYTH",
+ "username": "SCOTTY SMYTH",
"s": "a"
}
},
{
- "_id": 988366,
+ "_id": 960037,
"_score": 1,
"_source": {
- "username": "LAVONIA SMYTH",
+ "username": "EDGAR SMYTH",
"s": "a"
}
},
{
- "_id": 986580,
+ "_id": 958857,
"_score": 1,
"_source": {
- "username": "DARNELL SMYTH",
+ "username": "MIRTHA SMYTH",
"s": "a"
}
},
{
- "_id": 986231,
+ "_id": 957495,
"_score": 1,
"_source": {
- "username": "EDYTH BLODGETT",
+ "username": "SIXTA SMYTH",
"s": "a"
}
},
{
- "_id": 985595,
+ "_id": 953233,
"_score": 1,
"_source": {
- "username": "SETH BACON",
+ "username": "LAWRENCE SMYTH",
"s": "a"
}
},
{
- "_id": 982423,
+ "_id": 951184,
"_score": 1,
"_source": {
- "username": "SETH CHATMAN",
+ "username": "WILDA SMYTH",
"s": "a"
}
},
{
- "_id": 981115,
+ "_id": 944283,
"_score": 1,
"_source": {
- "username": "KATIE SMYTH",
+ "username": "HONG SMYTH",
"s": "a"
}
},
{
- "_id": 979541,
+ "_id": 936733,
"_score": 1,
"_source": {
- "username": "EDYTHE CLAYTON",
+ "username": "BEV SMYTH",
"s": "a"
}
},
{
- "_id": 979093,
+ "_id": 935940,
"_score": 1,
"_source": {
- "username": "SETH BURROUGHS",
+ "username": "CRISTY SMYTH",
"s": "a"
}
},
{
- "_id": 978812,
+ "_id": 929866,
"_score": 1,
"_source": {
- "username": "EDYTH GULLEY",
+ "username": "TEQUILA SMYTH",
"s": "a"
}
},
{
- "_id": 975994,
+ "_id": 915866,
"_score": 1,
"_source": {
- "username": "SETH PARHAM",
+ "username": "AMANDA SMYTH",
"s": "a"
}
},
{
- "_id": 975841,
+ "_id": 908615,
"_score": 1,
"_source": {
- "username": "ALEIDA SMYTH",
+ "username": "AMBERLY SMYTH",
"s": "a"
}
}
]
}
}
––– input –––
curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"layouts":"us,ru"}}' | jq -M
––– output –––
[
{
- "total": 10,
+ "total": 0,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
- ],
+ ]
- "data": [
+ }
- {
+ ]
- "query": "jordan"
- },
- {
- "query": "joseph"
- },
- {
- "query": "jose"
- },
- {
- "query": "joe"
- },
- {
- "query": "john"
- },
- {
- "query": "joy"
- },
- {
- "query": "joyce"
- },
- {
- "query": "joey"
- },
- {
- "query": "jon"
- },
- {
- "query": "major"
- }
- ]
- }
- ]
––– input –––
curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"append":1}}' | jq -M
––– output –––
[
{
- "total": 10,
+ "total": 0,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
- ],
+ ]
- "data": [
+ }
- {
+ ]
- "query": "jordan"
- },
- {
- "query": "joseph"
- },
- {
- "query": "jose"
- },
- {
- "query": "joe"
- },
- {
- "query": "john"
- },
- {
- "query": "joy"
- },
- {
- "query": "joyce"
- },
- {
- "query": "joey"
- },
- {
- "query": "jon"
- },
- {
- "query": "major"
- }
- ]
- }
- ]
––– input –––
curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"prepend":0}}' | jq -M
––– output –––
OK
––– input –––
curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"prepend":1,"append":1}}' | jq -M
––– output –––
[
{
- "total": 10,
+ "total": 0,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
- ],
+ ]
- "data": [
+ }
- {
+ ]
- "query": "jordan"
- },
- {
- "query": "joseph"
- },
- {
- "query": "jose"
- },
- {
- "query": "joe"
- },
- {
- "query": "john"
- },
- {
- "query": "joy"
- },
- {
- "query": "joyce"
- },
- {
- "query": "joey"
- },
- {
- "query": "jon"
- },
- {
- "query": "major"
- }
- ]
- }
- ]
––– input –––
curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":1,"layouts":"","append":1,"prepend":1}}' | jq -M
––– output –––
[
{
- "total": 10,
+ "total": 0,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
- ],
+ ]
- "data": [
+ }
- {
+ ]
- "query": "jordan"
- },
- {
- "query": "joseph"
- },
- {
- "query": "jose"
- },
- {
- "query": "joe"
- },
- {
- "query": "john"
- },
- {
- "query": "joy"
- },
- {
- "query": "joyce"
- },
- {
- "query": "joey"
- },
- {
- "query": "jon"
- },
- {
- "query": "major"
- }
- ]
- }
- ]
––– input –––
curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":2,"layouts":"us,ru","append":1,"prepend":1,"expansion_len":2}}' | jq -M
––– output –––
[
{
- "total": 6,
+ "total": 0,
"error": "",
"warning": "",
"columns": [
{
"query": {
"type": "string"
}
}
- ],
+ ]
- "data": [
+ }
- {
+ ]
- "query": "jose"
- },
- {
- "query": "joe"
- },
- {
- "query": "john"
- },
- {
- "query": "joy"
- },
- {
- "query": "joey"
- },
- {
- "query": "jon"
- }
- ]
- }
- ]
––– input –––
curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"gr","options":{"fuzziness":1,"layouts":"ru","append":1,"prepend":1}}' | jq -M
––– output –––
OK |
Contributor
Windows test results 3 files 3 suites 29m 53s ⏱️ Results for commit 656f5db. ♻️ This comment has been updated with latest results. |
Collaborator
Author
|
RAM use stats for the current branch compared to the master branch: The command used for the test is: |
Collaborator
Author
|
Another test with more typical token (English dictionary words), which also includes waiting for the table to finish optimizing: 
Radix: Master: |
Collaborator
Author
|
Another test that merges long chunks made up of many random string which causes lots of infixes. 
|
Collaborator
Author
|
As discussed:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Master branch:
This branch: