Tantivy Search

This library is designed to integrate tantivy into DingoDB.

Development

All FFI (Foreign Function Interface) functions are exposed in lib.rs. Developers need to regenerate the header file after making any changes to the relevant code:

cbindgen . -o include/tantivy_search_cbindgen.h --config cbindgen.toml
cxxbridge src/lib.rs --header > include/tantivy_search_cxx.h

Developers can use gen-cxx-header.sh to do this too.

If developers do not add, delete, or modify the names of FFI functions, there is no need to execute the above command.

If developers do not have cbindgen or cxxbridge, install like this:

cargo install cbindgen
cargo install cxxbridge-cmd

Work in progress

• Added unit tests in C++ to test Rust FFI functions.
• Add additional unit tests within Rust FFI functions.
• Refactor the tantivy_search code using object-oriented principles.

How to build?

You can use cargo to build this library, use this command:

cargo build --release

If you need to test FFI function in C++, run:

mkdir build
cd build && cmake ..
make -j

You can use vscode or other compilers to make the build process more elegant.

How to test?

Test in Rust:

cargo test

Here is an example to run unit test in C++:

cd build/tests/unit_test
./unit_test

Credits

We give special thanks for these open-source projects, upon which we have developed:

• Tantivy - A library for full-text search.
• tantivy-search - A library is designed to integrate tantivy into ClickHouse and MyScaleDB.
• cang-jie - A Chinese tokenizer for tantivy, based on jieba-rs.