Hybrid search

[marcel-vesely-kinit]

Important note: Let's try to reuse already existing ElasticSearch functionality first rather than re-implementing and storing text/sparse vectors within the Milvus collections to avoid redundancy.

• Use an embedding model for generating sparse vectors (e.g., BM25)
  • Text processing pipeline required (to convert text into a list of terms)
  • Specify vocabulary (inverted index)
• Incorporate a sparse embedding into the collections
• Implement Hybrid Search
  • Perform dense and sparse search separately
  • Use Reranker

More info can be found on:

• https://milvus.io/docs/multi-vector-search.md (Hybrid Search)
• https://milvus.io/docs/full-text-search.md (Full text search)