Skip to content

Commit 40aaf8c

Browse files
committed
edits
1 parent bf852a5 commit 40aaf8c

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

articles/ai-studio/concepts/evaluation-metrics-built-in.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ The risk and safety metrics draw on insights gained from our previous Large Lang
109109
- Direct attack jailbreak
110110
- Protected material content
111111

112-
You can measure these risk and safety metrics on your own data or test dataset through redteaming or on a synthetic test dataset generated by [our adversarial simulator](../how-to/develop/simulator-interaction-data.md). This will output an annotated test dataset with content risk severity levels (very low, low, medium, or high) and [view your results in Azure AI ](../how-to/evaluate-results.md), which provide you with overall defect rate across whole test dataset and instance view of each content risk label and reasoning.
112+
You can measure these risk and safety metrics on your own data or test dataset through redteaming or on a synthetic test dataset generated by [our adversarial simulator](../how-to/develop/simulator-interaction-data.md#generate-adversarial-simulations-for-safety-evaluation). This will output an annotated test dataset with content risk severity levels (very low, low, medium, or high) and [view your results in Azure AI ](../how-to/evaluate-results.md), which provide you with overall defect rate across whole test dataset and instance view of each content risk label and reasoning.
113113

114114
### Evaluating jailbreak vulnerability
115115

@@ -123,9 +123,9 @@ We support evaluating vulnerability towards the following types of jailbreak att
123123
1. Baseline adversarial test dataset
124124
2. Adversarial test dataset with direct attack jailbreak injections in the first turn.
125125

126-
You can do this with functionality and attack datasets generated with the [direct attack simulator](../how-to/develop/simulator-interaction-data.md) with the same randomization seed. Then you can evaluate jailbreak vulnerability by comparing results from content safety evaluators between the two test dataset's aggregate scores for each safety evaluator. A direct attack jailbreak defect is detected when there's presence of content harm response detected in the second direct attack injected dataset when there was none or lower severity detected in the first control dataset.
126+
You can do this with functionality and attack datasets generated with the [direct attack simulator](../how-to/develop/simulator-interaction-data.md#simulating-jailbreak-attacks) with the same randomization seed. Then you can evaluate jailbreak vulnerability by comparing results from content safety evaluators between the two test dataset's aggregate scores for each safety evaluator. A direct attack jailbreak defect is detected when there's presence of content harm response detected in the second direct attack injected dataset when there was none or lower severity detected in the first control dataset.
127127

128-
*Evaluating indirect attack* is an AI-assisted metric and doesn't require comparative measurement like evaluating direct attacks. Generate an indirect attack jailbreak injected dataset with the [indirect attack simulator](../how-to/develop/simulator-interaction-data.md) then evaluate with the `IndirectAttackEvaluator`.
128+
*Evaluating indirect attack* is an AI-assisted metric and doesn't require comparative measurement like evaluating direct attacks. Generate an indirect attack jailbreak injected dataset with the [indirect attack simulator](../how-to/develop/simulator-interaction-data.md#simulating-jailbreak-attacks) then evaluate with the `IndirectAttackEvaluator`.
129129

130130
> [!NOTE]
131131
> AI-assisted risk and safety metrics are hosted by Azure AI Studio safety evaluations back-end service and is only available in the following regions: East US 2, France Central, UK South, Sweden Central. Protected Material evaluation is only available in East US 2.

articles/ai-studio/how-to/develop/simulator-interaction-data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -233,7 +233,7 @@ azure_ai_project = {
233233
> [!NOTE]
234234
> Currently adversarial simulation, which uses the Azure AI safety evaluation service, is only available in the following regions: East US 2, France Central, UK South, Sweden Central.
235235
236-
## Specify target callback to simulate against - adversarial simulator
236+
## Specify target callback to simulate against for adversarial simulator
237237

238238
You can bring any application endpoint to the adversarial simulator. `AdversarialSimulator` class supports sending service-hosted queries and receiving responses with a callback function, as defined below. The `AdversarialSimulator` adheres to the [OpenAI's messages protocol](https://platform.openai.com/docs/api-reference/messages/object#messages/object-content).
239239

0 commit comments

Comments
 (0)