Skip to content

Commit 6cf972c

Browse files
authored
fix: agent goal accuracy (#1583)
1 parent 6d114e5 commit 6cf972c

File tree

2 files changed

+9
-1
lines changed

2 files changed

+9
-1
lines changed

docs/concepts/metrics/available_metrics/agents.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@ sample = MultiTurnSample(
9797
)
9898

9999
scorer = ToolCallAccuracy()
100+
scorer.llm = your_llm
100101
await scorer.multi_turn_ascore(sample)
101102
```
102103

@@ -146,7 +147,8 @@ sample = MultiTurnSample(user_input=[
146147
reference="Table booked at one of the chinese restaurants at 8 pm")
147148

148149
scorer = AgentGoalAccuracyWithReference()
149-
await metric.multi_turn_ascore(sample)
150+
scorer.llm = your_llm
151+
await scorer.multi_turn_ascore(sample)
150152

151153
```
152154

src/ragas/metrics/_goal_accuracy.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,9 @@ class AgentGoalAccuracyWithReference(MetricWithLLM, MultiTurnMetric):
114114
)
115115
max_retries: int = 1
116116

117+
async def _ascore(self, row: t.Dict, callbacks: Callbacks) -> float:
118+
raise NotImplementedError
119+
117120
async def _multi_turn_ascore(
118121
self,
119122
sample: MultiTurnSample,
@@ -153,6 +156,9 @@ class AgentGoalAccuracyWithoutReference(MetricWithLLM, MultiTurnMetric):
153156
)
154157
max_retries: int = 1
155158

159+
async def _ascore(self, row: t.Dict, callbacks: Callbacks) -> float:
160+
raise NotImplementedError
161+
156162
async def _multi_turn_ascore(
157163
self,
158164
sample: MultiTurnSample,

0 commit comments

Comments
 (0)