Docs for FastEmbed Ranker

[anakin87]

This component was introduced in #1178

It's pretty similar to TransformersSimilarityRanker, but this runs fast on CPU.

I can provide more details.

[anakin87]

Draft 👇

---

[block:parameters]
{
"data": {
"h-0": "",
"h-1": "",
"0-0": "Most common position in a pipeline",
"0-1": "In a query pipeline, after a component that returns a list of documents such as a Retriever ",
"1-0": "Mandatory init variables",
"1-1": "None",
"2-0": "Mandatory run variables",
"2-1": "“documents”: A list of documents \n \n”query”: A query string",
"3-0": "Output variables",
"3-1": "“documents”: A list of documents",
"4-0": "API reference",
"4-1": "Rankers",
"5-0": "GitHub link",
"5-1": "https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed"
},
"cols": 2,
"rows": 6,
"align": [
"left",
"left"
]
}
[/block]

Overview

FastembedRanker ranks the given documents based on how similar they are to the given query. It uses cross-encoder models supported by FastEmbed.
Based on ONXX Runtime, FastEmbed provides a fast experience on standard CPU machines.

FastembedRanker is most useful in query pipelines, such as a retrieval-augmented generation (RAG) pipeline or a document search pipeline, to ensure the retrieved documents are ordered by relevance. You can use it after a Retriever (such as the InMemoryEmbeddingRetriever) to improve the search results. When using FastembedRanker with a Retriever, consider setting the Retriever's top_k to a small number. This way, the Ranker will have fewer documents to process, which can help make your pipeline faster.

By default, this component uses the Xenova/ms-marco-MiniLM-L-6-v2 model, but it's flexible. You can switch to a different model by adjusting the model parameter when initializing the Ranker. For details on different initialization settings, check out the API reference for this component.

Compatible models

You can find the original models in the FastEmbed documentation.

Installation

To start using this integration with Haystack, install the package with:

pip install fastembed-haystack

Parameters

You can set the path where the model will be stored in a cache directory. Also, you can set the number of threads a single onnxruntime session can use.

cache_dir= "/your_cacheDirectory"
ranker = FastembedRanker(
	model="Xenova/ms-marco-MiniLM-L-6-v2",
	cache_dir=cache_dir,
	threads=2
)

If you want to use the data parallel encoding, you can set the parameters parallel and batch_size.

• If parallel > 1, data-parallel encoding will be used. This is recommended for offline encoding of large datasets.
• If parallel is 0, use all available cores.
• If None, don't use data-parallel processing; use default onnxruntime threading instead.

Usage

On its own

This example uses FastembedRanker to rank two simple documents. To run the Ranker, pass a query, provide the documents, and set the number of documents to return in the top_k parameter.

from haystack import Document
from haystack_integrations.components.rankers.fastembed import FastembedRanker

docs = [Document(content="Paris"), Document(content="Berlin")]

ranker = FastembedRanker()
ranker.warm_up()

ranker.run(query="City in France", documents=docs, top_k=1)

In a pipeline

Below is an example of a pipeline that retrieves documents from an InMemoryDocumentStore based on keyword search (using InMemoryBM25Retriever). It then uses the FastembedRanker to rank the retrieved documents according to their similarity to the query. The pipeline uses the default settings of the Ranker.

from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.rankers.fastembed import FastembedRanker

docs = [
    Document(content="Paris is in France"),
    Document(content="Berlin is in Germany"),
    Document(content="Lyon is in France"),
]
document_store = InMemoryDocumentStore()
document_store.write_documents(docs)

retriever = InMemoryBM25Retriever(document_store=document_store)
ranker = FastembedRanker()

document_ranker_pipeline = Pipeline()
document_ranker_pipeline.add_component(instance=retriever, name="retriever")
document_ranker_pipeline.add_component(instance=ranker, name="ranker")

document_ranker_pipeline.connect("retriever.documents", "ranker.documents")

query = "Cities in France"
res = document_ranker_pipeline.run(data={"retriever": {"query": query, "top_k": 3}, "ranker": {"query": query, "top_k": 2}})

👍 top_k parameter

In the example above, the top_k values for the Retriever and the Ranker are different. The Retriever's top_k specifies how many documents it returns. The Ranker then orders these documents.

You can set the same or a smaller top_k value for the Ranker. The Ranker's top_k is the number of documents it returns (if it's the last component in the pipeline) or forwards to the next component. In the pipeline example above, the Ranker is the last component, so the output you get when you run the pipeline are the top two documents, as per the Ranker's top_k.

Adjusting the top_k values can help you optimize performance. In this case, a smaller top_k value of the Retriever means fewer documents to process for the Ranker, which can speed up the pipeline.

[dfokina]

Published: https://docs.haystack.deepset.ai/docs/fastembedranker