You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-responsible-ai-dashboard.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -43,7 +43,7 @@ The Responsible AI dashboard is accompanied by a [PDF scorecard](how-to-responsi
43
43
44
44
The Responsible AI dashboard brings together, in a comprehensive view, various new and pre-existing tools. The dashboard integrates these tools with [Azure Machine Learning CLI v2, Azure Machine Learning Python SDK v2](concept-v2.md), and [Azure Machine Learning studio](overview-what-is-azure-machine-learning.md#studio). The tools include:
45
45
46
-
-[Data explorer](concept-data-analysis.md), to understand and explore your dataset distributions and statistics.
46
+
-[Data analysis](concept-data-analysis.md), to understand and explore your dataset distributions and statistics.
47
47
-[Model overview and fairness assessment](concept-fairness-ml.md), to evaluate the performance of your model and evaluate your model's group fairness issues (how your model's predictions affect diverse groups of people).
48
48
-[Error analysis](concept-error-analysis.md), to view and understand how errors are distributed in your dataset.
49
49
-[Model interpretability](how-to-machine-learning-interpretability.md) (importance values for aggregate and individual features), to understand your model's predictions and how those overall and individual predictions are made.
@@ -83,7 +83,7 @@ The following table describes when to use Responsible AI dashboard components to
83
83
| Identify | Error analysis | The error analysis component helps you get a deeper understanding of model failure distribution and quickly identify erroneous cohorts (subgroups) of data. <br><br> The capabilities of this component in the dashboard come from the [Error Analysis](https://erroranalysis.ai/) package.|
84
84
| Identify | Fairness analysis | The fairness component defines groups in terms of sensitive attributes such as sex, race, and age. It then assesses how your model predictions affect these groups and how you can mitigate disparities. It evaluates the performance of your model by exploring the distribution of your prediction values and the values of your model performance metrics across the groups. <br><br>The capabilities of this component in the dashboard come from the [Fairlearn](https://fairlearn.org/) package. |
85
85
| Identify | Model overview | The model overview component aggregates model assessment metrics in a high-level view of model prediction distribution for better investigation of its performance. This component also enables group fairness assessment by highlighting the breakdown of model performance across sensitive groups. |
86
-
| Diagnose | Data explorer|The data explorer visualizes datasets based on predicted and actual outcomes, error groups, and specific features. You can then identify issues of overrepresentation and underrepresentation, along with seeing how data is clustered in the dataset. |
86
+
| Diagnose | Data analysis|Data analysis visualizes datasets based on predicted and actual outcomes, error groups, and specific features. You can then identify issues of overrepresentation and underrepresentation, along with seeing how data is clustered in the dataset. |
87
87
| Diagnose | Model interpretability | The interpretability component generates human-understandable explanations of the predictions of a machine learning model. It provides multiple views into a model's behavior: <br> - Global explanations (for example, which features affect the overall behavior of a loan allocation model) <br> - Local explanations (for example, why an applicant's loan application was approved or rejected) <br><br> The capabilities of this component in the dashboard come from the [InterpretML](https://interpret.ml/) package. |
88
88
| Diagnose | Counterfactual analysis and what-if| This component consists of two functionalities for better error diagnosis: <br> - Generating a set of examples in which minimal changes to a particular point alter the model's prediction. That is, the examples show the closest data points with opposite model predictions. <br> - Enabling interactive and custom what-if perturbations for individual data points to understand how the model reacts to feature changes. <br> <br> The capabilities of this component in the dashboard come from the [DiCE](https://github.com/interpretml/DiCE) package. |
89
89
@@ -108,7 +108,7 @@ Exploratory data analysis, causal inference, and counterfactual analysis capabil
108
108
109
109
These components of the Responsible AI dashboard support responsible decision-making:
110
110
111
-
-**Data explorer**: You can reuse the data explorer component here to understand data distributions and to identify overrepresentation and underrepresentation. Data exploration is a critical part of decision making, because it isn't feasible to make informed decisions about a cohort that's underrepresented in the data.
111
+
-**Data analysis**: You can reuse the data analysis component here to understand data distributions and to identify overrepresentation and underrepresentation. Data exploration is a critical part of decision making, because it isn't feasible to make informed decisions about a cohort that's underrepresented in the data.
112
112
-**Causal inference**: The causal inference component estimates how a real-world outcome changes in the presence of an intervention. It also helps construct promising interventions by simulating feature responses to various interventions and creating rules to determine which population cohorts would benefit from a particular intervention. Collectively, these functionalities allow you to apply new policies and effect real-world change.
113
113
114
114
The capabilities of this component come from the [EconML](https://github.com/Microsoft/EconML) package, which estimates heterogeneous treatment effects from observational data via machine learning.
@@ -142,14 +142,14 @@ Need some inspiration? Here are some examples of how the dashboard's components
142
142
143
143
| Responsible AI dashboard flow | Use case |
144
144
|-------------------------------|----------|
145
-
| Model overview > error analysis > data explorer| To identify model errors and diagnose them by understanding the underlying data distribution |
146
-
| Model overview > fairness assessment > data explorer| To identify model fairness issues and diagnose them by understanding the underlying data distribution |
145
+
| Model overview > error analysis > data analysis| To identify model errors and diagnose them by understanding the underlying data distribution |
146
+
| Model overview > fairness assessment > data analysis| To identify model fairness issues and diagnose them by understanding the underlying data distribution |
147
147
| Model overview > error analysis > counterfactuals analysis and what-if | To diagnose errors in individual instances with counterfactual analysis (minimum change to lead to a different model prediction) |
148
-
| Model overview > data explorer| To understand the root cause of errors and fairness issues introduced via data imbalances or lack of representation of a particular data cohort |
148
+
| Model overview > data analysis| To understand the root cause of errors and fairness issues introduced via data imbalances or lack of representation of a particular data cohort |
149
149
| Model overview > interpretability | To diagnose model errors through understanding how the model has made its predictions |
150
-
| Data explorer > causal inference | To distinguish between correlations and causations in the data or decide the best treatments to apply to get a positive outcome |
150
+
| Data analysis > causal inference | To distinguish between correlations and causations in the data or decide the best treatments to apply to get a positive outcome |
151
151
| Interpretability > causal inference | To learn whether the factors that the model has used for prediction-making have any causal effect on the real-world outcome|
152
-
| Data explorer > counterfactuals analysis and what-if | To address customers' questions about what they can do next time to get a different outcome from an AI system|
152
+
| Data analysis > counterfactuals analysis and what-if | To address customers' questions about what they can do next time to get a different outcome from an AI system|
153
153
154
154
## People who should use the Responsible AI dashboard
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-responsible-ai-dashboard.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -168,7 +168,7 @@ Select the **Heat map** tab to switch to a different view of the error in the da
168
168
5.**Cells**: Represents a cohort of the dataset, with filters applied, and the percentage of errors out of the total number of data points in the cohort. A blue outline indicates selected cells, and the darkness of red represents the concentration of failures.
169
169
6.**Prediction path (filters)**: Lists the filters placed over the full dataset for each selected cohort.
170
170
171
-
### Model overview
171
+
### Model overview and fairness metrics
172
172
173
173
The model overview component provides a comprehensive set of performance and fairness metrics for evaluating your model, along with key performance disparity metrics along specified features and dataset cohorts.
174
174
@@ -219,7 +219,7 @@ With the data analysis component, the **Table view** pane shows you a table view
219
219
220
220
The **Chart view** panel shows you aggregate and individual plots of datapoints. You can analyze data statistics along the x-axis and y-axis by using filters such as predicted outcome, dataset features, and error groups. This view helps you understand overrepresentation and underrepresentation in your dataset.
221
221
222
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard/data-analysis-table-view.png" alt-text="Screenshot of the dashboard, showing the data explorer." lightbox= "./media/how-to-responsible-ai-dashboard/data-analysis-table-view.png":::
222
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard/data-analysis-table-view.png" alt-text="Screenshot of the dashboard, showing the data analysis." lightbox= "./media/how-to-responsible-ai-dashboard/data-analysis-table-view.png":::
223
223
224
224
1.**Select a dataset cohort to explore**: Specify which dataset cohort from your list of cohorts you want to view data statistics for.
225
225
2.**X-axis**: Displays the type of value being plotted horizontally. Modify the values by selecting the button to open a side panel.
@@ -228,7 +228,7 @@ The **Chart view** panel shows you aggregate and individual plots of datapoints.
228
228
229
229
By selecting the **Individual data points** option under **Chart type**, you can shift to a disaggregated view of the data with the availability of a color axis.
230
230
231
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard/data-analysis-individual-datapoints.png" alt-text="Screenshot of the dashboard, showing the data explorer with the 'Individual data points' option selected." lightbox= "./media/how-to-responsible-ai-dashboard/data-analysis-individual-datapoints.png":::
231
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard/data-analysis-individual-datapoints.png" alt-text="Screenshot of the dashboard, showing the data analysis with the 'Individual data points' option selected." lightbox= "./media/how-to-responsible-ai-dashboard/data-analysis-individual-datapoints.png":::
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-responsible-ai-scorecard.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,9 +38,9 @@ The Responsible AI scorecard is a PDF summary of key insights from your Responsi
38
38
39
39
:::image type="content" source="./media/how-to-responsible-ai-scorecard/scorecard-summary.png" alt-text="Screenshot of the model summary on the Responsible AI scorecard PDF.":::
40
40
41
-
The data explorer segment shows you characteristics of your data, because any model story is incomplete without a correct understanding of your data:
41
+
The data analysis segment shows you characteristics of your data, because any model story is incomplete without a correct understanding of your data:
42
42
43
-
:::image type="content" source="./media/how-to-responsible-ai-scorecard/scorecard-data-explorer.png" alt-text="Screenshot of the data explorer on the Responsible AI scorecard PDF.":::
43
+
:::image type="content" source="./media/how-to-responsible-ai-scorecard/scorecard-data-explorer.png" alt-text="Screenshot of the data analysis on the Responsible AI scorecard PDF.":::
44
44
45
45
The model performance segment displays your model's most important metrics and characteristics of your predictions and how well they satisfy your desired target values:
0 commit comments