Skip to content

Commit 3c2b3f6

Browse files
committed
Metric doc update, links in other docs and preview
1 parent cdf1fc6 commit 3c2b3f6

File tree

6 files changed

+41
-65
lines changed

6 files changed

+41
-65
lines changed

articles/ai-foundry/concepts/ai-red-teaming-agent.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ ms.author: lagayhar
1111
author: lgayhardt
1212
---
1313

14-
# AI Red Teaming Agent
14+
# AI Red Teaming Agent (preview)
1515

1616
[!INCLUDE [feature-preview](../includes/feature-preview.md)]
1717

articles/ai-foundry/concepts/evaluation-approach-gen-ai.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ The pre-production stage acts as a final quality check, reducing the risk of dep
5959
- Simulators and AI red teaming agent: If you don’t have evaluation data (test data), Azure AI [Evaluation SDK’s simulators](..//how-to/develop/simulator-interaction-data.md) can help by generating topic-related or adversarial queries. These simulators test the model’s response to situation-appropriate or attack-like queries (edge cases).
6060
- The [adversarial simulator](../how-to/develop/simulator-interaction-data.md#generate-adversarial-simulations-for-safety-evaluation) injects queries that mimic potential safety risks or security attacks such as or attempt jailbreaks, helping identify limitations and preparing the model for unexpected conditions.
6161
- [Context-appropriate simulators](../how-to/develop/simulator-interaction-data.md#generate-synthetic-data-and-simulate-non-adversarial-tasks) generate typical, relevant conversations you’d expect from users to test quality of responses. With context-appropriate simulators you can assess metrics such as groundedness, relevance, coherence, and fluency of generated responses.
62-
- AI red teaming agent simulates adversarial attacks against your proactively stress-test models and applications against broad range of safety and security attacks using Microsoft’s open framework for Python Risk Identification Tool or [PyRIT](https://github.com/Azure/PyRIT). Automated scans using the AI red teaming agent enhances pre-production risk assessment by systematically testing AI applications for vulnerabilities. This process involves simulated attack scenarios to identify weaknesses in model responses before real-world deployment. By running AI red teaming scans, you can detect and mitigate potential security risks before deployment. This tool is recommended to be used in conjunction with human-in-the-loop processes such as conventional AI red teaming probing to help accelerate risk identification and aid in the assessment by a human expert.
62+
- [AI red teaming agent](../how-to/develop/run-scans-ai-red-teaming-agent.md) simulates adversarial attacks against your proactively stress-test models and applications against broad range of safety and security attacks using Microsoft’s open framework for Python Risk Identification Tool or [PyRIT](https://github.com/Azure/PyRIT). Automated scans using the AI red teaming agent enhances pre-production risk assessment by systematically testing AI applications for vulnerabilities. This process involves simulated attack scenarios to identify weaknesses in model responses before real-world deployment. By running AI red teaming scans, you can detect and mitigate potential security risks before deployment. This tool is recommended to be used in conjunction with human-in-the-loop processes such as conventional AI red teaming probing to help accelerate risk identification and aid in the assessment by a human expert.
6363

6464
Alternatively, you can also use [Azure AI Foundry’s evaluation widget](../how-to/evaluate-generative-ai-app.md) for testing your generative AI applications.
6565

@@ -91,6 +91,7 @@ Cheat sheet:
9191
## Related content
9292

9393
- [Evaluate your generative AI apps via the playground](../how-to/evaluate-prompts-playground.md)
94+
- [Run automated scans with the AI red teaming agent to assess safety and security risks](../how-to/develop/run-scans-ai-red-teaming-agent.md)
9495
- [Evaluate your generative AI apps with the Azure AI Foundry SDK or portal](../how-to/evaluate-generative-ai-app.md)
9596
- [Evaluation and monitoring metrics for generative AI](evaluation-metrics-built-in.md)
9697
- [Transparency Note for Azure AI Foundry safety evaluations](safety-evaluations-transparency-note.md)

0 commit comments

Comments
 (0)