Don't understand the sentence_embedding implementation

[jgcb00]

Hi,

I don't get how the code is working,
So the main idea is to replace the forward method of the model to use Ctranslate2 Model instead of the classic forward method of sentence transformers.

But in the case I want to get the sentence embeddings in sentence_transformers there is this line 181 :

embeddings = out_features[output_value]

output_value = 'sentence_embedding' by default, so it must have been calculated in the forward method because everything before is in an if statement.

The forward method, is supercharged by this one, so I should get somewhere in the forward method :
features["sentence_embedding"] as it is returning features, but it's just not there, so I don't get how you managed to make it works....

Thanks for your time

[michaelfeil]

I recommend you to check out: https://github.com/michaelfeil/infinity for better sentence transformer support!

[michaelfeil]

Otherwise, this might work: https://github.com/michaelfeil/hf-hub-ctranslate2/blob/main/hf_hub_ctranslate2/ct2_sentence_transformers.py