Running jina-embeddings-v2-base-en embedding model LOCALLY

[Danana1]

I want to use Jinal embedding model: jina-embeddings-v2-base-en LOCALLY.
I can run it locally in this form:
model = SentenceTransformer("jinaai/jina-embeddings-v2-base-code", trust_remote_code=True)
but using the Jinambeddings API requires an API KEY, meaning this will not run locally on my machine:

from langchain_community.embeddings import JinaEmbeddings
text_embeddings = JinaEmbeddings(
    jina_api_key="jina_*", model_name="jina-embeddings-v2-base-en"
)

I was wondering how can I use this embedding model locally, like I do with JinaEmbeddings or HuggingFace API

[dosubot[bot]]

There isn't a built-in way to use JinaEmbeddings locally in LangChain without an API key—JinaEmbeddings is designed to work with the remote Jina API and always requires a key, with no documented support for local or offline use in the official code or docs (source).

However, since you can run the model locally with SentenceTransformer, you can use the HuggingFaceEmbeddings or SentenceTransformerEmbeddings integration in LangChain to wrap your local model, just like you would with any other HuggingFace-compatible embedding model. This approach doesn't require an API key and works entirely offline.

If you want a LangChain Embeddings interface for your local model, you can do something like:

from langchain_community.embeddings import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(
    model_name="jinaai/jina-embeddings-v2-base-en",
    model_kwargs={"trust_remote_code": True}
)

Or, if you want more control, you can subclass the Embeddings interface and wrap your SentenceTransformer instance directly.

Let me know if you want a code example for that!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[dosubot[bot]]

This error happens because the Jina model uses a custom config class (JinaBertConfig), but something in the HuggingFace/SentenceTransformer loading pipeline is trying to use the standard BertConfig instead, causing a mismatch. This is a known issue with some models that use trust_remote_code and custom architectures.

To fix this, make sure you are using the exact model name ("jinaai/jina-embeddings-v2-base-en") and set trust_remote_code=True in model_kwargs. However, even with this, some versions of transformers or sentence-transformers may not handle custom configs well when called through wrappers like HuggingFaceEmbeddings.

If you still get the error, the most reliable workaround is to load the model directly with SentenceTransformer and then wrap it in a custom embedding class for LangChain, bypassing HuggingFaceEmbeddings entirely. For example:

from sentence_transformers import SentenceTransformer
from langchain.embeddings.base import Embeddings

class JinaLocalEmbeddings(Embeddings):
    def __init__(self, model_name="jinaai/jina-embeddings-v2-base-en"):
        self.model = SentenceTransformer(model_name, trust_remote_code=True)

    def embed_documents(self, texts):
        return self.model.encode(texts).tolist()

    def embed_query(self, text):
        return self.model.encode([text])[0].tolist()

This approach avoids the config_class mismatch by not going through the HuggingFaceEmbeddings wrapper, which passes extra config handling to transformers and can trigger this error with custom models. This pattern is also suggested in community discussions for similar issues with custom HuggingFace models (example).

If you want to keep using HuggingFaceEmbeddings, double-check your transformers and sentence-transformers versions, as some combinations are more robust with trust_remote_code models than others. But the custom wrapper above is the most reliable fix.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[Danana1]

OK! I think we made some progress. This is the class I created since there were other functions which were abstract:

class JinaLocalEmbeddings(BaseEmbedding):
    def __init__(self, model_name="jinaai/jina-embeddings-v2-base-en"):
        super().__init__()
        self.model = SentenceTransformer(model_name, trust_remote_code=True)

    def embed_documents(self, texts):
        return self.model.encode(texts).tolist()

    def embed_query(self, text):
        return self.model.encode([text])[0].tolist()
    
    def _get_text_embedding_batch(self, texts):
        return self._model._get_text_embedding_batch(texts)

    def _get_query_embedding_batch(self, queries):
        return self._model._get_query_embedding_batch(queries)

    def _get_text_embedding(self, text):
        return self._get_text_embedding_batch([text])[0]

    def _get_query_embedding(self, query):
        return self._get_query_embedding_batch([query])[0]

    async def _aget_query_embedding(self, query):
        return self._get_query_embedding(query)

    async def _aget_text_embedding(self, text):
        return self._get_text_embedding(text)

However, when this is called:
self.model = SentenceTransformer(model_name, trust_remote_code=True)

I get the following exception:

 File "...\Lib\site-packages\pydantic\main.py", line 926, in __setattr__
    raise ValueError(f'"{self.__class__.__name__}" object has no field "{name}"')
ValueError: "JinaLocalEmbeddings" object has no field "model"

[dosubot[bot]]

The error happens because your JinaLocalEmbeddings class is being treated as a Pydantic model, which restricts setting arbitrary attributes like self.model unless they're declared as fields. However, LangChain's Embeddings classes do not require or expect Pydantic—standard Python attribute assignment works fine.

To fix this, make sure your class inherits directly from langchain_core.embeddings.Embeddings (or langchain.embeddings.base.Embeddings), not from a Pydantic BaseModel or any class that mixes in Pydantic. If you have a BaseEmbedding that inherits from Pydantic's BaseModel, that's the source of the problem.

Here's a minimal working example:

from langchain_core.embeddings import Embeddings
from sentence_transformers import SentenceTransformer

class JinaLocalEmbeddings(Embeddings):
    def __init__(self, model_name="jinaai/jina-embeddings-v2-base-en"):
        self.model = SentenceTransformer(model_name, trust_remote_code=True)

    def embed_documents(self, texts):
        return self.model.encode(texts).tolist()

    def embed_query(self, text):
        return self.model.encode([text])[0].tolist()

This pattern matches how custom embedding classes are implemented in LangChain without Pydantic interference (example, example). If you need to keep other abstract methods, just add them as normal Python methods.

If you must use Pydantic for some reason, you can declare model as a PrivateAttr, but that's not necessary or recommended for typical LangChain embedding classes.

Let me know if you need help adapting this to your use case!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[Danana1]

Can you explain what is the difference between:
embed_documents and embed_query?

[dosubot[bot]]

embed_documents is for embedding a list of texts (like multiple documents or sentences) and returns a list of embeddings—one for each input. embed_query is for embedding a single text (like a search query) and returns just one embedding.

In practice, you’d use embed_documents when you want to encode a batch of texts (for example, all your documents for retrieval), and embed_query when you want to encode a single query (for example, a user’s search input). The methods are often implemented similarly under the hood, but the distinction allows you to treat queries and documents differently if needed (for example, applying different instructions or prompts) source.

So: use embed_documents for bulk/batch embedding, and embed_query for one-off or search-time embedding.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}