Skip to content

Commit fd5e805

Browse files
Jeff-67jeff yang
andauthored
Improve efficiency in factual correctness for precision mode (#1578)
### Description This pull request optimizes the `_single_turn_ascore` function by conditionally skipping unnecessary computations when the mode is set to "precision." Previously, the function calculated `response_reference` regardless of the scoring mode, which consumed resources and potentially increased processing costs. By avoiding the `response_reference` calculation in "precision" mode, this update aims to enhance speed and reduce computational expenses. ### Changes Made - Modified `_single_turn_ascore` function: - **Conditionally calculated `response_reference`**: The `response_reference` is now computed only when the mode is not "precision." --------- Co-authored-by: jeff yang <[email protected]>
1 parent 2319dce commit fd5e805

File tree

1 file changed

+12
-4
lines changed

1 file changed

+12
-4
lines changed

src/ragas/metrics/_factual_correctness.py

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -244,13 +244,21 @@ async def _single_turn_ascore(
244244
reference_response = await self.verify_claims(
245245
premise=reference, hypothesis_list=response_claims, callbacks=callbacks
246246
)
247-
response_reference = await self.verify_claims(
248-
premise=response, hypothesis_list=reference_claims, callbacks=callbacks
249-
)
247+
248+
249+
if self.mode != "precision":
250+
response_reference = await self.verify_claims(
251+
premise=response, hypothesis_list=reference_claims, callbacks=callbacks
252+
)
253+
else:
254+
response_reference = np.array([])
250255

251256
true_positives = sum(reference_response)
252257
false_positives = sum(~reference_response)
253-
false_negatives = sum(~response_reference)
258+
if self.mode != "precision":
259+
false_negatives = sum(~response_reference)
260+
else:
261+
false_negatives = 0
254262

255263
if self.mode == "precision":
256264
score = true_positives / (true_positives + false_positives + 1e-8)

0 commit comments

Comments
 (0)