Skip to content

Commit 23c86a3

Browse files
authored
Detailed error message for noise sensitivity metric (#1986)
Absence of any of the required inputs **(user_input, response, reference or retrieved_contexts)** in the test sample results in **KeyError**. For example, the absence of **user_input** in the test sample results in the error message **"KeyError: user_input"**. The error message **"KeyError: user_input"** is too abstract. I included the following lines of code in the **_ascore()** function definition which will display a clear error message in the absence of any of the required inputs. ``` if "reference" not in row or not row["reference"]: raise ValueError("reference is missing in the test sample. Please add reference to the test sample.") if "user_input" not in row or not row["user_input"]: raise ValueError("user_input is missing in the test sample. Please add user_input to the test sample.") if "response" not in row or not row["response"]: raise ValueError("response is missing in the test sample. Please add response to the test sample.") if "retrieved_contexts" not in row or not row["retrieved_contexts"]: raise ValueError("retrieved_contexts is missing in the test sample. Please add retrieved_contexts to the test sample.") ```
1 parent c11f530 commit 23c86a3

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

src/ragas/metrics/_noise_sensitivity.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,18 @@ async def _ascore(self, row: t.Dict, callbacks: Callbacks) -> float:
126126
"""
127127
assert self.llm is not None, "LLM is not set"
128128

129+
if "reference" not in row or not row["reference"]:
130+
raise ValueError("reference is missing in the test sample. Please add reference to the test sample.")
131+
132+
if "user_input" not in row or not row["user_input"]:
133+
raise ValueError("user_input is missing in the test sample. Please add user_input to the test sample.")
134+
135+
if "response" not in row or not row["response"]:
136+
raise ValueError("response is missing in the test sample. Please add response to the test sample.")
137+
138+
if "retrieved_contexts" not in row or not row["retrieved_contexts"]:
139+
raise ValueError("retrieved_contexts is missing in the test sample. Please add retrieved_contexts to the test sample.")
140+
129141
gt_statements = await self._decompose_answer_into_statements(
130142
row["reference"], row["user_input"], callbacks
131143
)

0 commit comments

Comments
 (0)