Skip to content

Commit e0867d1

Browse files
Handle embeddings for empty strings in AnswerSimilarity class (#994)
Fixes: #996 Some embedding models do not work with empty strings (for example Gemini) and return an error. The error does not appear with OpenAI which returns an embeddings even for an empty string. Proposed resolution: replace empty strings with `" "`
1 parent 4c36cb9 commit e0867d1

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

src/ragas/metrics/_answer_similarity.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,10 @@ async def _ascore(self: t.Self, row: t.Dict, callbacks: Callbacks) -> float:
6969
ground_truth = t.cast(str, row["reference"])
7070
answer = t.cast(str, row["response"])
7171

72+
# Handle embeddings for empty strings
73+
ground_truth = ground_truth or " "
74+
answer = answer or " "
75+
7276
if self.is_cross_encoder and isinstance(self.embeddings, HuggingfaceEmbeddings):
7377
raise NotImplementedError(
7478
"async score [ascore()] not implemented for HuggingFace embeddings"

0 commit comments

Comments
 (0)