Sentence transformers and embedding output are different

[raulod]

I followed the same steps as in #5801

In my case, the output of SentenseTransformer and embedding is different. What I am doing wrong? The output from the embedding and server are same though.

model = SentenceTransformer(r"sentence-transformers/paraphrase-MiniLM-L6-v2")
sentences = ["What is your age?"]
embeddings = model.encode(sentences)
print(embeddings)

8.89059186e-01  7.43575931e-01  7.86601365e-01 -8.80633481e-03   -2.86287606e-01  8.21958408e-02 
...
 -1.10238581e-03  1.25207350e-01 -4.75726455e-01  1.67811707e-01 6.75540805e-01  1.02285840e-01 

llama.cpp

convert_hf_to_gguf.py ~/.cache/torch/sentence_transformers/sentence-transformers_paraphrase-MiniLM-L6-v2/ --outtype f32

./llama-embedding -m paraphrase-MiniLM-L6-23M-v2-F32.gguf  -p "What is your age?"

0.120811  0.100893  0.106925 -0.001156 -0.038927  0.011093 -0.017685  0.045368  
...
-0.000201  0.016992 -0.064531  0.022764  0.091856  0.013816 -0.028264 -0.005593 

[raulod]

Looking forward to some inputs. thanks

[danbev]

I'm not familiar with the SentenceTransformer but the llama-embedding example uses Euclidean/L2 normalization by default:

--embd-normalize N                      normalisation for embendings (default: 2) (-1=none, 0=max absolute
                                        int16, 1=taxicab, 2=euclidean, >2=p-norm)

Perhaps you can try to disable the normalization in llama-embedding like this:

./llama-embedding -m paraphrase-MiniLM-L6-23M-v2-F32.gguf -p "What is your age?" --embd-normalize -1 --verbose-prompt

[raulod]

thank you!

[danbev]

If this works/helps would you be able to mark this question as answered?