Why is there a separate interface for Huggingface embeddings and specifically Huggingface BGE Embeddings?

[JamalRahman]

BGE embeddings hosted on Huggingface are runnable via sentence-transformers, which is the underlying mechanism used in Langchain.

Yet in Langchain there is a separate class for interacting with BGE embeddings; langchain.embeddings.huggingface.HuggingFaceBgeEmbeddings versus [langchain.embeddings.huggingface.HuggingFaceEmbeddings

I haven't found any documentation anywhere that explains the reason for a BGE-specific interface.

Is there some form of incompatibility? If not it seems misleading!

[JamalRahman]

Decided to investigate this myself.

The difference is that BGE works by having a prompt (query_instruction) that gets prepended to your search query.
The non-BGE Huggingface interface doesn't support this query instruction. the BGE Huggingface interface does support this query instruction parameter.

These two are functionally equivalent:

non_bge = HuggingFaceEmbeddings(
    model_name="BAAI/bge-small-en-v1.5",
    model_kwargs={"device": "cpu"},
    encode_kwargs={"normalize_embeddings": True},
)
bge = HuggingFaceBgeEmbeddings(
    model_name="BAAI/bge-small-en-v1.5",
    model_kwargs={"device": "cpu"},
    encode_kwargs={"normalize_embeddings": True},
    query_instruction=""

)

I'll create a PR for updating the docs to explain this.

[mkhludnev]

come up with PR #22405 please check