Skip to content

Commit 895f9c0

Browse files
authored
docs: change setting llm (#1639)
1 parent 092a0c8 commit 895f9c0

File tree

2 files changed

+18
-15
lines changed

2 files changed

+18
-15
lines changed

docs/concepts/metrics/available_metrics/general_purpose.md

Lines changed: 17 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ sample = SingleTurnSample(
2222
scorer = AspectCritic(
2323
name="maliciousness",
2424
definition="Is the submission intended to harm, deceive, or exploit users?",
25+
llm=evaluator_llm
26+
2527
)
26-
scorer.llm = openai_model
2728
await scorer.single_turn_ascore(sample)
2829
```
2930

@@ -45,8 +46,10 @@ sample = SingleTurnSample(
4546
scorer = AspectCritic(
4647
name="correctness",
4748
definition="Is the response factually similar to the reference?",
49+
llm=evaluator_llm
50+
4851
)
49-
scorer.llm = openai_model
52+
5053
await scorer.single_turn_ascore(sample)
5154

5255
```
@@ -83,8 +86,10 @@ sample = SingleTurnSample(
8386
response="The Eiffel Tower is located in Paris.",
8487
)
8588

86-
scorer = SimpleCriteriaScoreWithoutReference(name="course_grained_score", definition="Score 0 to 5 for correctness")
87-
scorer.llm = openai_model
89+
scorer = SimpleCriteriaScoreWithoutReference(name="course_grained_score",
90+
definition="Score 0 to 5 for correctness",
91+
llm=evaluator_llm
92+
)
8893
await scorer.single_turn_ascore(sample)
8994
```
9095

@@ -101,8 +106,10 @@ sample = SingleTurnSample(
101106
reference="The Eiffel Tower is located in Egypt"
102107
)
103108

104-
scorer = SimpleCriteriaScoreWithReference(name="course_grained_score", definition="Score 0 to 5 by similarity")
105-
scorer.llm = openai_model
109+
scorer = SimpleCriteriaScoreWithReference(name="course_grained_score",
110+
definition="Score 0 to 5 by similarity",
111+
llm=evaluator_llm)
112+
106113
await scorer.single_turn_ascore(sample)
107114
```
108115

@@ -130,8 +137,7 @@ rubrics = {
130137
"score4_description": "The response is mostly accurate and aligns well with the ground truth, with only minor issues or missing details.",
131138
"score5_description": "The response is fully accurate, aligns completely with the ground truth, and is clear and detailed.",
132139
}
133-
scorer = RubricsScoreWithReference(rubrics=)
134-
scorer.llm = openai_model
140+
scorer = RubricsScoreWithReference(rubrics=rubrics, llm=evaluator_llm)
135141
await scorer.single_turn_ascore(sample)
136142
```
137143

@@ -148,8 +154,7 @@ sample = SingleTurnSample(
148154
response="The Eiffel Tower is located in Paris.",
149155
)
150156

151-
scorer = RubricsScoreWithoutReference()
152-
scorer.llm = openai_model
157+
scorer = RubricsScoreWithoutReference(rubrics=rubrics, llm=evaluator_llm)
153158
await scorer.single_turn_ascore(sample)
154159
```
155160

@@ -181,8 +186,7 @@ SingleTurnSample(
181186
}
182187
)
183188

184-
scorer = InstanceRubricsWithReference()
185-
scorer.llm = openai_model
189+
scorer = InstanceRubricsWithReference(llm=evaluator_llm)
186190
await scorer.single_turn_ascore(sample)
187191
```
188192

@@ -208,7 +212,6 @@ SingleTurnSample(
208212
}
209213
)
210214

211-
scorer = InstanceRubricsScoreWithoutReference()
212-
scorer.llm = openai_model
215+
scorer = InstanceRubricsScoreWithoutReference(llm=evaluator_llm)
213216
await scorer.single_turn_ascore(sample)
214217
```

docs/concepts/metrics/available_metrics/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Each metric are essentially paradigms that are designed to evaluate a particular
1616

1717
## Agents or Tool use cases
1818

19-
- [Topic adherence](topic_adherence.md)
19+
- [Topic adherence](agents.md#topic_adherence)
2020
- [Tool call Accuracy](agents.md#tool-call-accuracy)
2121
- [Agent Goal Accuracy](agents.md#agent-goal-accuracy)
2222

0 commit comments

Comments
 (0)