You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: update AnswerCorrectness to collections-based API
- Add collections-based API example with LLM and embeddings setup
- Move legacy evaluate() example to Legacy Metrics API section
- Add deprecation warning for legacy API
- Include synchronous usage note
- Tested example code and verified it works correctly
Note: Embeddings are required because default weights [0.75, 0.25]
include semantic similarity. See issue #2408 for making embeddings
optional when weights[1] == 0.
If you prefer synchronous code, you can use the `.score()` method instead of `.ascore()`:
49
+
50
+
```python
51
+
result = scorer.score(
52
+
user_input="When was the first super bowl?",
53
+
response="The first superbowl was held on Jan 15, 1967",
54
+
reference="The first superbowl was held on January 15, 1967"
55
+
)
56
+
```
33
57
34
58
### Calculation
35
59
@@ -57,3 +81,26 @@ Next, we calculate the semantic similarity between the generated answer and the
57
81
58
82
Once we have the semantic similarity, we take a weighted average of the semantic similarity and the factual similarity calculated above to arrive at the final score. You can adjust this weightage by modifying the `weights` parameter.
59
83
84
+
## Legacy Metrics API
85
+
86
+
The following examples use the legacy metrics API pattern. For new projects, we recommend using the collections-based API shown above.
87
+
88
+
!!! warning "Deprecation Timeline"
89
+
This API will be deprecated in version 0.4 and removed in version 1.0. Please migrate to the collections-based API shown above.
90
+
91
+
### Example with Dataset
92
+
93
+
```python
94
+
from datasets import Dataset
95
+
from ragas.metrics import answer_correctness
96
+
from ragas import evaluate
97
+
98
+
data_samples = {
99
+
'question': ['When was the first super bowl?', 'Who won the most super bowls?'],
100
+
'answer': ['The first superbowl was held on Jan 15, 1967', 'The most super bowls have been won by The New England Patriots'],
101
+
'ground_truth': ['The first superbowl was held on January 15, 1967', 'The New England Patriots have won the Super Bowl a record six times']
0 commit comments