You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: detectors/llm_judge/README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# LLM Judge Detector
2
2
3
-
The LLM Judge detector integrates the [vLLM Judge](https://github.com/trustyai-explainability/vllm_judge) into the Guardrails Detector ecosystem. Please refer [llm_judge_examples](docs/llm_judge_examples.md) for usage details.
3
+
The LLM Judge detector integrates the [vLLM Judge](https://github.com/trustyai-explainability/vllm_judge) into the Guardrails Detector ecosystem. Please refer [llm_judge_examples](../../docs/llm_judge_examples.md) for usage details.
Instead of writing complex prompts let us use pre-defined prompts for common scenarios. vLLM-Judge library provides various builtin metrics (which are pre-defined prompts) including moderation (`safety`, `toxicity`, `bias`), general purpose (`helpfulness`, `clarity`, `conciseness`, `relevance`, `coherence`) and others (like code quality, content quality). Let's use one of the pre-defined metrics from vLLM-Judge, eg., `safety`.
29
29
30
-
The `safety` metric evaluates the content safety across multiple dimensions like physical harm, psycological harm, social harm and vulnerable groups.
30
+
The `safety` metric evaluates the content safety across multiple dimensions like physical harm, psychological harm, social harm and vulnerable groups.
31
31
32
32
**Request:**
33
33
```bash
@@ -134,9 +134,9 @@ curl -s -X POST \
134
134
]
135
135
```
136
136
137
-
We get pretty ok results where model uses positive label (like 'True') and higher scores (like 1.0) for positive instances i.e, that satisfy the criteria and similarly negative label ('FAIL') and lower score (0.2) for negative instances i.e, that does not satisfy the criteria.
137
+
We get pretty ok results where model uses positive label (like 'True') and higher scores (like 1.0) for positive instances i.e., those that satisfy the criteria and similarly negative label ('FAIL') and lower score (0.2) for negative instances i.e., those that do not satisfy the criteria.
138
138
139
-
But how to specifically say which labels to use and how to assign scores? This is where the `rubric` parameter comes in.
139
+
But how do you specify which labels to use and how to assign scores? This is where the `rubric` parameter comes in.
140
140
141
141
#### Example 3: Custom Labels and Scoring with Rubrics
142
142
@@ -265,7 +265,7 @@ Below is the full list of parameters that can be passed to `detector_params` to
265
265
-`template_vars`: Variable mapping to substitute in templates
266
266
-`template_engine`: Template engine to use ('format' or 'jinja2'), default is 'format'
267
267
-`system_prompt`: Custom system message to take full control of the evaluator LLM persona
268
-
-`examples`: Few-shot examples. List of JSON objects, each JSON represents an example and must contain `content`, `score`, and `reasoning` fields and `reasoning` fields
268
+
-`examples`: Few-shot examples. List of JSON objects, each JSON represents an example and must contain `content`, `score`, and `reasoning` fields
0 commit comments