Error when using hkunlp/instructor-large as a retriver

[gooseillo]

Hi all,

I'm trying to use this model hkunlp/instructor-large in my retriver to calculate embeddings for my ES index but I get this error-

Code-

from haystack.nodes import EmbeddingRetriever
from haystack.nodes import EmbeddingRetriever


retriever = EmbeddingRetriever(
    document_store=document_store,
     embedding_model="hkunlp/instructor-base",
     model_format = "sentence_transformers"   
)

Error - 
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[10], line 4
      1 from haystack.nodes import EmbeddingRetriever
----> 4 retriever = EmbeddingRetriever(
      5     document_store=document_store,
      6     # embedding_model="sentence-transformers/all-MiniLM-L6-v2",
      7      embedding_model="hkunlp/instructor-base",
      8      model_format = "sentence_transformers"
      9      
     10 )

File [c:\Users\russe\anaconda3\envs\haystackllm\lib\site-packages\haystack\nodes\base.py:46](file:///C:/Users/russe/anaconda3/envs/haystackllm/lib/site-packages/haystack/nodes/base.py:46), in exportable_to_yaml..wrapper_exportable_to_yaml(self, *args, **kwargs)
     43         self._component_config["params"][k] = v
     45 # Call the actuall __init__ function with all the arguments
---> 46 init_func(self, *args, **kwargs)

File [c:\Users\russe\anaconda3\envs\haystackllm\lib\site-packages\haystack\nodes\retriever\dense.py:1580](file:///C:/Users/russe/anaconda3/envs/haystackllm/lib/site-packages/haystack/nodes/retriever/dense.py:1580), in EmbeddingRetriever.__init__(self, embedding_model, document_store, model_version, use_gpu, batch_size, max_seq_len, model_format, pooling_strategy, emb_extraction_layer, top_k, progress_bar, devices, use_auth_token, scale_score, embed_meta_fields, api_key, azure_api_version, azure_base_url, azure_deployment_name)
   1567 if (
   1568     self.embedding_model.startswith("sentence-transformers")
   1569     and model_format
   1570     and model_format != "sentence_transformers"
   1571 ):
   1572     logger.warning(
   1573         "You seem to be using a Sentence Transformer embedding model but 'model_format' is set to '%s'."
   1574         " You may need to set model_format='sentence_transformers' to ensure correct loading of model."
   (...)
   1577         self.model_format,
   1578     )
-> 1580 self.embedding_encoder = _EMBEDDING_ENCODERS[self.model_format](retriever=self)
   1581 self.embed_meta_fields = embed_meta_fields

File [c:\Users\russe\anaconda3\envs\haystackllm\lib\site-packages\haystack\nodes\retriever\_embedding_encoder.py:128](file:///C:/Users/russe/anaconda3/envs/haystackllm/lib/site-packages/haystack/nodes/retriever/_embedding_encoder.py:128), in _SentenceTransformersEmbeddingEncoder.__init__(self, retriever)
    124     from haystack.utils.import_utils import _optional_component_not_installed
    126     _optional_component_not_installed(__name__, "sentence", ie)
--> 128 self.embedding_model = SentenceTransformer(
    129     retriever.embedding_model, device=str(retriever.devices[0]), use_auth_token=retriever.use_auth_token
    130 )
    131 self.batch_size = retriever.batch_size
    132 self.embedding_model.max_seq_length = retriever.max_seq_len

File [c:\Users\russe\anaconda3\envs\haystackllm\lib\site-packages\sentence_transformers\SentenceTransformer.py:95](file:///C:/Users/russe/anaconda3/envs/haystackllm/lib/site-packages/sentence_transformers/SentenceTransformer.py:95), in SentenceTransformer.__init__(self, model_name_or_path, modules, device, cache_folder, use_auth_token)
     87         snapshot_download(model_name_or_path,
     88                             cache_dir=cache_folder,
     89                             library_name='sentence-transformers',
     90                             library_version=__version__,
     91                             ignore_files=['flax_model.msgpack', 'rust_model.ot', 'tf_model.h5'],
     92                             use_auth_token=use_auth_token)
     94 if os.path.exists(os.path.join(model_path, 'modules.json')):    #Load as SentenceTransformer model
---> 95     modules = self._load_sbert_model(model_path)
     96 else:   #Load with AutoModel
     97     modules = self._load_auto_model(model_path)

File [c:\Users\russe\anaconda3\envs\haystackllm\lib\site-packages\sentence_transformers\SentenceTransformer.py:840](file:///C:/Users/russe/anaconda3/envs/haystackllm/lib/site-packages/sentence_transformers/SentenceTransformer.py:840), in SentenceTransformer._load_sbert_model(self, model_path)
    838 for module_config in modules_config:
    839     module_class = import_from_string(module_config['type'])
--> 840     module = module_class.load(os.path.join(model_path, module_config['path']))
    841     modules[module_config['name']] = module
    843 return modules

File [c:\Users\russe\anaconda3\envs\haystackllm\lib\site-packages\sentence_transformers\models\Pooling.py:120](file:///C:/Users/russe/anaconda3/envs/haystackllm/lib/site-packages/sentence_transformers/models/Pooling.py:120), in Pooling.load(input_path)
    117 with open(os.path.join(input_path, 'config.json')) as fIn:
    118     config = json.load(fIn)
--> 120 return Pooling(**config)

TypeError: Pooling.__init__() got an unexpected keyword argument 'pooling_mode_weightedmean_tokens'

Does any one know how I can fix this?

[bogdankostic]

Hi @gooseillo, it seems that this model is not supported with the sentence-transformers library, see also this discussion on the model's page. As we make use of sentence-transformers under the hood, it's currently not possible to run this model in Haystack.

[gooseillo]

Thanks @bogdankostic . That what I had suspected.
Is there any other way I could make use of that model for embeddings within the Haystack retriver? Can I somehow integrate the code from https://github.com/HKUNLP/instructor-embedding to create embeddings for the ES index and use Haystack to retrieve them?

This instructor model is much better at technical and scientific documents.

[anakin87]

I too am interested in this discussion.
@bogdankostic Is there interest in supporting these embeddings in the future?

[bogdankostic]

There's an open issue for adding instructor embeddings, but we haven't prioritized it yet, so there's no timeline when and if we will implement this.

[gooseillo]

@bogdankostic in that issue he mentioned he tried this to make it work.

Describe alternatives you've considered
At the moment, the only possible way to solve it is to override EmbeddingRetriever but it seems like a wrong solution for the long term.

Can you explain what he meant by overriding the embedding retriver?

[bogdankostic]

I'm not entirely sure what he meant with overriding the EmbeddingRetriever, but I guess he created his own custom InstructorRetriever that inherits from the base class DenseRetriever.