Sort out relevance functions

[klaudialemiec]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

I would like to enhance relevance score functions. I would like to unify relevance scores and make them independent from vectorstores. Relevance scores functions are implemented in base class VectorStores. The functions require distance as parameter. The problem is that same of vectorstores return distances (i.e Weaviate), and some returns similarity scores (i.e. Pinecone while calling similarity_search_with_score methods. I propose to unify scores returned by similarity_search_with_score (only distances or only similarity scores) add pass the output as input to relevance_scores_fn. This is partially done (i.e. Pinecone vectorstore class implement its own cosine_relevance score, but it takes cosine similarity as input rather than cosine distance).

The second point it that distance can't always be standarized to range 0-1 (for example euclidean or L2 distance). I would like to sort out what distances/similarity metrics are returned by popular vectorstores and handle the possible metrics.

Does it make sense?

Motivation

I think relevance functions are not clear

Proposal (If applicable)

No response