Support for Hybrid Search in Databricks retriever (query_type="hybrid")

[reslleygabriel]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Include a new argument for the similarity search methods (similarity_score_threshold retrieval) or a new type of search "hybrid_search"

Databricks documentation

Motivation

Hybrid keyword-similarity search combines vector-based embedding search with traditional keyword-based search techniques. This approach matches exact words in the query while also using a vector-based similarity search to capture the semantic relationships and context of the query.

By integrating these two techniques, hybrid keyword-similarity search retrieves documents that contain not only the exact keywords but also those that are conceptually similar, providing more comprehensive and relevant search results. This method is particularly useful in RAG applications where source data has unique keywords such as SKUs or identifiers that are not well suited to pure similarity search.

Proposal (If applicable)

Include query_type="hybrid" as a parameter in the as_retriever method (as search_type and search_kwargs)

[reslleygabriel]

It seems to be addressed in this PR, and we are currently waiting for the merge.