Feat/2286 conversational rag search

[KlimTodrik]

No description provided.

[github-actions]

clt

❌ CLT tests in test/clt-tests/http-interface/
✅ OK: 11
❌ Failed: 1
⏳ Duration: 56s
👉 Check Action Results for commit 3e38cc2

Failed tests:

🔧 Edit failed tests in UI:

• Edit test/clt-tests/http-interface/cli-json-endpoint.rec

test/clt-tests/http-interface/cli-json-endpoint.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd $SEARCHD_FLAGS > /dev/null; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE t (id INT, value TEXT, value_attr STRING) min_infix_len = '3' min_prefix_len = '3'; INSERT INTO t VALUES (1, 'example', 'example'), (2, 'test', 'test');"
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/cli_json?show%20version"
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/cli_json?show%20buddy%20plugins"
––– output –––
- [{"total":27,"error":"","warning":"","columns":[{"Package":{"type":"string"}},{"Plugin":{"type":"string"}},{"Version":{"type":"string"}},{"Type":{"type":"string"}},{"Info":{"type":"string"}}],"data":[{"Package":"manticoresoftware/buddy-plugin-empty-string","Plugin":"empty-string","Version":"%{VERSION}","Type":"core","Info":"Handles empty queries, which can occur when trimming comments or dealing with specific SQL protocol instructions in comments that are not supported"},{"Package":"manticoresoftware/buddy-plugin-backup","Plugin":"backup","Version":"%{VERSION}","Type":"core","Info":"BACKUP sql statement"},{"Package":"manticoresoftware/buddy-plugin-emulate-elastic","Plugin":"emulate-elastic","Version":"%{VERSION}","Type":"core","Info":"Emulates some Elastic queries and generates responses as if they were made by ES"},{"Package":"manticoresoftware/buddy-plugin-fuzzy","Plugin":"fuzzy","Version":"%{VERSION}","Type":"core","Info":"Fuzzy search plugin. It helps to find the best match for a given query."},{"Package":"manticoresoftware/buddy-plugin-create-table","Plugin":"create-table","Version":"%{VERSION}","Type":"core","Info":"Enables tables copying; handles CREATE TABLE statements with MySQL options not supported by Manticore"},{"Package":"manticoresoftware/buddy-plugin-create-cluster","Plugin":"create-cluster","Version":"%{VERSION}","Type":"core","Info":"Enable CREATE CLUSTER IF NOT EXISTS statements"},{"Package":"manticoresoftware/buddy-plugin-drop","Plugin":"drop","Version":"%{VERSION}","Type":"core","Info":"Handles DROP statements with MySQL options not supported by Manticore"},{"Package":"manticoresoftware/buddy-plugin-insert","Plugin":"insert","Version":"%{VERSION}","Type":"core","Info":"Auto schema support. When an insert operation is performed and the table does not exist, it creates it with data types auto-detection"},{"Package":"manticoresoftware/buddy-plugin-alias","Plugin":"alias","Version":"%{VERSION}","Type":"core","Info":""},{"Package":"manticoresoftware/buddy-plugin-select","Plugin":"select","Version":"%{VERSION}","Type":"core","Info":"Various SELECTs handlers needed for mysqldump and other software support, mostly aiming to work similarly to MySQL"},{"Package":"manticoresoftware/buddy-plugin-show","Plugin":"show","Version":"%{VERSION}","Type":"core","Info":"Various \"show\" queries handlers, for example, `show queries`, `show fields`, `show full tables`, etc"},{"Package":"manticoresoftware/buddy-plugin-plugin","Plugin":"plugin","Version":"%{VERSION}","Type":"core","Info":"Core logic for plugin support and helpers. Also handles `create buddy plugin`, `delete buddy plugin`, and `show buddy plugins`"},{"Package":"manticoresoftware/buddy-plugin-test","Plugin":"test","Version":"%{VERSION}","Type":"core","Info":"Test plugin, used exclusively for tests"},{"Package":"manticoresoftware/buddy-plugin-alter-column","Plugin":"alter-column","Version":"%{VERSION}","Type":"core","Info":"Enables adding/dropping table fields(columns)"},{"Package":"manticoresoftware/buddy-plugin-alter-distributed-table","Plugin":"alter-distributed-table","Version":"%{VERSION}","Type":"core","Info":"Enables alter for distributed tables"},{"Package":"manticoresoftware/buddy-plugin-alter-rename-table","Plugin":"alter-rename-table","Version":"%{VERSION}","Type":"core","Info":"Enables alter table rename"},{"Package":"manticoresoftware/buddy-plugin-modify-table","Plugin":"modify-table","Version":"%{VERSION}","Type":"core","Info":"Assists in standardizing options in create and alter table statements to show option=1 for integers."},{"Package":"manticoresoftware/buddy-plugin-knn","Plugin":"knn","Version":"%{VERSION}","Type":"core","Info":"Enables KNN by document id"},{"Package":"manticoresoftware/buddy-plugin-replace","Plugin":"replace","Version":"%{VERSION}","Type":"core","Info":"Enables partial replaces"},{"Package":"manticoresoftware/buddy-plugin-queue","Plugin":"queue","Version":"%{VERSION}","Type":"core","Info":""},{"Package":"manticoresoftware/buddy-plugin-sharding","Plugin":"sharding","Version":"%{VERSION}","Type":"core","Info":"Enables sharded tables."},{"Package":"manticoresoftware/buddy-plugin-update","Plugin":"update","Version":"%{VERSION}","Type":"core","Info":"Handles UPDATE statements sent by MySQL tools making possible for them to update full-text fields"},{"Package":"manticoresoftware/buddy-plugin-autocomplete","Plugin":"autocomplete","Version":"%{VERSION}","Type":"core","Info":"Autocomplete plugin that offers suggestions based on the starting query"},{"Package":"manticoresoftware/buddy-plugin-cli-table","Plugin":"cli-table","Version":"%{VERSION}","Type":"core","Info":"/cli endpoint based on /cli_json - outputs query result as a table"},{"Package":"manticoresoftware/buddy-plugin-distributed-insert","Plugin":"distributed-insert","Version":"%{VERSION}","Type":"core","Info":"The plugin enables data insertion into distributed sharded tables."},{"Package":"manticoresoftware/buddy-plugin-truncate","Plugin":"truncate","Version":"%{VERSION}","Type":"core","Info":"Handles TRUNCATE statements on distributed tables"},{"Package":"manticoresoftware/buddy-plugin-metrics","Plugin":"metrics","Version":"%{VERSION}","Type":"core","Info":"Returns Prometheus metrics"}]}]
+ [{"total":28,"error":"","warning":"","columns":[{"Package":{"type":"string"}},{"Plugin":{"type":"string"}},{"Version":{"type":"string"}},{"Type":{"type":"string"}},{"Info":{"type":"string"}}],"data":[{"Package":"manticoresoftware/buddy-plugin-empty-string","Plugin":"empty-string","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Handles empty queries, which can occur when trimming comments or dealing with specific SQL protocol instructions in comments that are not supported"},{"Package":"manticoresoftware/buddy-plugin-backup","Plugin":"backup","Version":"3.38.0-9-g36a88d","Type":"core","Info":"BACKUP sql statement"},{"Package":"manticoresoftware/buddy-plugin-emulate-elastic","Plugin":"emulate-elastic","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Emulates some Elastic queries and generates responses as if they were made by ES"},{"Package":"manticoresoftware/buddy-plugin-fuzzy","Plugin":"fuzzy","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Fuzzy search plugin. It helps to find the best match for a given query."},{"Package":"manticoresoftware/buddy-plugin-create-table","Plugin":"create-table","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enables tables copying; handles CREATE TABLE statements with MySQL options not supported by Manticore"},{"Package":"manticoresoftware/buddy-plugin-create-cluster","Plugin":"create-cluster","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enable CREATE CLUSTER IF NOT EXISTS statements"},{"Package":"manticoresoftware/buddy-plugin-drop","Plugin":"drop","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Handles DROP statements with MySQL options not supported by Manticore"},{"Package":"manticoresoftware/buddy-plugin-insert","Plugin":"insert","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Auto schema support. When an insert operation is performed and the table does not exist, it creates it with data types auto-detection"},{"Package":"manticoresoftware/buddy-plugin-alias","Plugin":"alias","Version":"3.38.0-9-g36a88d","Type":"core","Info":""},{"Package":"manticoresoftware/buddy-plugin-select","Plugin":"select","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Various SELECTs handlers needed for mysqldump and other software support, mostly aiming to work similarly to MySQL"},{"Package":"manticoresoftware/buddy-plugin-show","Plugin":"show","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Various \"show\" queries handlers, for example, `show queries`, `show fields`, `show full tables`, etc"},{"Package":"manticoresoftware/buddy-plugin-plugin","Plugin":"plugin","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Core logic for plugin support and helpers. Also handles `create buddy plugin`, `delete buddy plugin`, and `show buddy plugins`"},{"Package":"manticoresoftware/buddy-plugin-test","Plugin":"test","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Test plugin, used exclusively for tests"},{"Package":"manticoresoftware/buddy-plugin-alter-column","Plugin":"alter-column","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enables adding/dropping table fields(columns)"},{"Package":"manticoresoftware/buddy-plugin-alter-distributed-table","Plugin":"alter-distributed-table","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enables alter for distributed tables"},{"Package":"manticoresoftware/buddy-plugin-alter-rename-table","Plugin":"alter-rename-table","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enables alter table rename"},{"Package":"manticoresoftware/buddy-plugin-modify-table","Plugin":"modify-table","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Assists in standardizing options in create and alter table statements to show option=1 for integers."},{"Package":"manticoresoftware/buddy-plugin-knn","Plugin":"knn","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enables KNN by document id"},{"Package":"manticoresoftware/buddy-plugin-replace","Plugin":"replace","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enables partial replaces"},{"Package":"manticoresoftware/buddy-plugin-queue","Plugin":"queue","Version":"3.38.0-9-g36a88d","Type":"core","Info":""},{"Package":"manticoresoftware/buddy-plugin-sharding","Plugin":"sharding","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Enables sharded tables."},{"Package":"manticoresoftware/buddy-plugin-update","Plugin":"update","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Handles UPDATE statements sent by MySQL tools making possible for them to update full-text fields"},{"Package":"manticoresoftware/buddy-plugin-autocomplete","Plugin":"autocomplete","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Autocomplete plugin that offers suggestions based on the starting query"},{"Package":"manticoresoftware/buddy-plugin-cli-table","Plugin":"cli-table","Version":"3.38.0-9-g36a88d","Type":"core","Info":"/cli endpoint based on /cli_json - outputs query result as a table"},{"Package":"manticoresoftware/buddy-plugin-distributed-insert","Plugin":"distributed-insert","Version":"3.38.0-9-g36a88d","Type":"core","Info":"The plugin enables data insertion into distributed sharded tables."},{"Package":"manticoresoftware/buddy-plugin-truncate","Plugin":"truncate","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Handles TRUNCATE statements on distributed tables"},{"Package":"manticoresoftware/buddy-plugin-metrics","Plugin":"metrics","Version":"3.38.0-9-g36a88d","Type":"core","Info":"Returns Prometheus metrics"},{"Package":"manticoresoftware/buddy-plugin-conversational-rag","Plugin":"conversational-rag","Version":"3.38.0-9-g36a88d","Type":"core","Info":""}]}]
––– input –––
curl -s "http://localhost:9308/cli_json?select%20*%20from%20t%20where%20match('exmaple')%20option%20fuzzy=1"
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/cli_json?show%20fields%20from%20t"
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/cli_json?create%20table%20t_copy2%20like%20t"
––– output –––
OK
––– input –––
curl -s "http://localhost:9308/cli_json?select%20*%20from%20t%20limit%201"
––– output –––
OK

[github-actions]

🐳 Docker Images Pushed

Buddy Test-Kit Images:

• Branch image: ghcr.io/manticoresoftware/manticoresearch:test-kit-feat_2286-conversational-rag-search-buddy
• Commit image: ghcr.io/manticoresoftware/manticoresearch:test-kit-3e38cc2b-buddy

Base image used: ghcr.io/manticoresoftware/manticoresearch:test-kit-latest

These images contain the buddy code from this PR and can be used for testing.

Built from commit: 36a88d3