You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cognitive-services/openai/concepts/red-teaming.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,7 +25,7 @@ Microsoft has conducted red teaming exercises and implemented safety systems (in
25
25
- Identify and mitigate shortcomings in the existing default filters or mitigation strategies.
26
26
- Provide feedback on failures so we can make improvements.
27
27
28
-
Here's how you can get started in your process of red teaming LLMs. Advance planning is critical to a productive red teaming exercise.
28
+
Here is how you can get started in your process of red teaming LLMs. Advance planning is critical to a productive red teaming exercise.
29
29
30
30
## Getting started
31
31
@@ -41,13 +41,13 @@ Having red teamers with an adversarial mindset and security-testing experience i
41
41
42
42
**Remember that handling potentially harmful content can be mentally taxing.**
43
43
44
-
You'll need to take care of your red teamers, not only by limiting the amount of time they spend on an assignment, but also by letting them know they can opt out at any time. Also, avoid burnout by switching red teamers’ assignments to different focus areas.
44
+
You will need to take care of your red teamers, not only by limiting the amount of time they spend on an assignment, but also by letting them know they can opt out at any time. Also, avoid burnout by switching red teamers’ assignments to different focus areas.
45
45
46
46
### Planning your red teaming
47
47
48
48
#### Where to test
49
49
50
-
Because a system is developed using an LLM base model, you may need to test at several different layers:
50
+
Because a system is developed using a LLM base model, you may need to test at several different layers:
51
51
52
52
- The LLM base model with its [safety system](./content-filter.md) in place to identify any gaps that may need to be addressed in the context of your application system. (Testing is usually through an API endpoint.)
53
53
- Your application system. (Testing is usually through a UI.)
@@ -57,12 +57,12 @@ Because a system is developed using an LLM base model, you may need to test at s
57
57
58
58
Consider conducting iterative red teaming in at least two phases:
59
59
60
-
1. Open-ended red teaming, where red teamers are encouraged to discover various harms. This can help you develop a taxonomy of harms to guide further testing. Note that developing a taxonomy of undesired LLM outputs for your application system is crucial to being able to measure the success of specific mitigation efforts.
60
+
1. Open-ended red teaming, where red teamers are encouraged to discover a variety of harms. This can help you develop a taxonomy of harms to guide further testing. Note that developing a taxonomy of undesired LLM outputs for your application system is crucial to being able to measure the success of specific mitigation efforts.
61
61
2. Guided red teaming, where red teamers are assigned to focus on specific harms listed in the taxonomy while staying alert for any new harms that may emerge. Red teamers can also be instructed to focus testing on specific features of a system for surfacing potential harms.
62
62
63
63
Be sure to:
64
64
65
-
- Provide your red teamers with clear instructions for what harms or system features they'll be testing.
65
+
- Provide your red teamers with clear instructions for what harms or system features they will be testing.
66
66
- Give your red teamers a place for recording their findings. For example, this could be a simple spreadsheet specifying the types of data that red teamers should provide, including basics such as:
67
67
- The type of harm that was surfaced.
68
68
- The input prompt that triggered the output.
@@ -72,7 +72,7 @@ Be sure to:
72
72
73
73
### Reporting red teaming findings
74
74
75
-
You'll want to summarize and report red teaming top findings at regular intervals to key stakeholders, including teams involved in the measurement and mitigation of LLM failures so that the findings can inform critical decision making and prioritizations.
75
+
You will want to summarize and report red teaming top findings at regular intervals to key stakeholders, including teams involved in the measurement and mitigation of LLM failures so that the findings can inform critical decision making and prioritizations.
0 commit comments