You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: Perform exploratory data analysis to understand feature biases and imbalances by using the Responsible AI dashboard's data explorer.
4
+
description: Perform exploratory data analysis to understand feature biases and imbalances by using the Responsible AI dashboard's data analysis.
5
5
services: machine-learning
6
6
ms.service: machine-learning
7
7
ms.subservice: enterprise-readiness
8
8
ms.topic: how-to
9
9
ms.author: mesameki
10
10
author: mesameki
11
11
ms.reviewer: lagayhar
12
-
ms.date: 08/17/2022
12
+
ms.date: 11/09/2022
13
13
ms.custom: responsible-ml, event-tier1-build-2022
14
14
---
15
15
16
16
# Understand your datasets (preview)
17
17
18
-
Machine learning models "learn" from historical decisions and actions captured in training data. As a result, their performance in real-world scenarios is heavily influenced by the data they're trained on. When feature distribution in a dataset is skewed, it can cause a model to incorrectly predict data points that belong to an underrepresented group or to be optimized along an inappropriate metric.
18
+
Machine learning models "learn" from historical decisions and actions captured in training data. As a result, their performance in real-world scenarios is heavily influenced by the data they're trained on. When feature distribution in a dataset is skewed, it can cause a model to incorrectly predict data points that belong to an underrepresented group or to be optimized along an inappropriate metric.
19
19
20
20
For example, while a model was training an AI system for predicting house prices, the training set was representing 75 percent of newer houses that had less than median prices. As a result, it was much less accurate in successfully identifying more expensive historic houses. The fix was to add older and expensive houses to the training data and augment the features to include insights about historical value. That data augmentation improved results.
21
21
22
-
The data explorer component of the [Responsible AI dashboard](concept-responsible-ai-dashboard.md) helps visualize datasets based on predicted and actual outcomes, error groups, and specific features. It helps you identify issues of overrepresentation and underrepresentation and to see how data is clustered in the dataset. Data visualizations consist of aggregate plots or individual data points.
22
+
The data analysis component of the [Responsible AI dashboard](concept-responsible-ai-dashboard.md) helps visualize datasets based on predicted and actual outcomes, error groups, and specific features. It helps you identify issues of overrepresentation and underrepresentation and to see how data is clustered in the dataset. Data visualizations consist of aggregate plots or individual data points.
23
23
24
-
## When to use the data explorer
24
+
## When to use data analysis
25
25
26
-
Use the data explorer when you need to:
26
+
Use data analysis when you need to:
27
27
28
28
- Explore your dataset statistics by selecting different filters to slice your data into different dimensions (also known as cohorts).
29
29
- Understand the distribution of your dataset across different cohorts and feature groups.
@@ -33,5 +33,5 @@ Use the data explorer when you need to:
33
33
## Next steps
34
34
35
35
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-insights-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
36
-
- Explore the [supported data explorer visualizations](how-to-responsible-ai-dashboard.md#data-analysis) of the Responsible AI dashboard.
36
+
- Explore the [supported data analysis visualizations](how-to-responsible-ai-dashboard.md#data-analysis) of the Responsible AI dashboard.
37
37
- Learn how to generate a [Responsible AI scorecard](how-to-responsible-ai-scorecard.md) based on the insights observed in the Responsible AI dashboard.
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-fairness-ml.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -90,7 +90,7 @@ The Fairlearn open-source package provides two types of unfairness mitigation al
90
90
91
91
## Next steps
92
92
93
-
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-dashboard-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-dashboard-ui.md).
93
+
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-insights-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
94
94
- Explore the [supported model overview and fairness assessment visualizations](how-to-responsible-ai-dashboard.md#model-overview-and-fairness-metrics) of the Responsible AI dashboard.
95
95
- Learn how to generate a [Responsible AI scorecard](how-to-responsible-ai-scorecard.md) based on the insights observed in the Responsible AI dashboard.
96
96
- Learn how to use the components by checking out Fairlearn's [GitHub repository](https://github.com/fairlearn/fairlearn/), [user guide](https://fairlearn.github.io/main/user_guide/index.html), [examples](https://fairlearn.github.io/main/auto_examples/index.html), and [sample notebooks](https://github.com/fairlearn/fairlearn/tree/master/notebooks).
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-responsible-ai-dashboard.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ ms.topic: how-to
9
9
ms.author: mesameki
10
10
author: mesameki
11
11
ms.reviewer: lagayhar
12
-
ms.date: 11/07/2022
12
+
ms.date: 11/09/2022
13
13
ms.custom: responsible-ml, event-tier1-build-2022
14
14
---
15
15
@@ -174,5 +174,5 @@ The following people can use the Responsible AI dashboard, and its corresponding
174
174
175
175
## Next steps
176
176
177
-
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-dashboard-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-dashboard-ui.md).
177
+
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-insights-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
178
178
- Learn how to generate a [Responsible AI scorecard](concept-responsible-ai-scorecard.md) based on the insights observed on the Responsible AI dashboard.
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-responsible-ai.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ ms.topic: conceptual
9
9
ms.author: mesameki
10
10
author: mesameki
11
11
ms.reviewer: lagayhar
12
-
ms.date: 11/07/2022
12
+
ms.date: 11/09/2022
13
13
ms.custom: responsible-ai, event-tier1-build-2022
14
14
#Customer intent: As a data scientist, I want to learn what Responsible AI is and how I can use it in Azure Machine Learning.
15
15
---
@@ -57,9 +57,9 @@ The model interpretability component provides multiple views into a model's beha
57
57
58
58
-*Global explanations*. For example, what features affect the overall behavior of a loan allocation model?
59
59
-*Local explanations*. For example, why was a customer's loan application approved or rejected?
60
-
-*Model explanations for a selected cohort of data points*. For example, what features affect the overall behavior of a loan allocation model for low-income applicants?
60
+
-*Model explanations for a selected cohort of data points*. For example, what features affect the overall behavior of a loan allocation model for low-income applicants?
61
61
62
-
The counterfactual what-if component enables understanding and debugging a machine learning model in terms of how it reacts to feature changes and perturbations.
62
+
The counterfactual what-if component enables understanding and debugging a machine learning model in terms of how it reacts to feature changes and perturbations.
63
63
64
64
Azure Machine Learning also supports a [Responsible AI scorecard](./how-to-responsible-ai-scorecard.md). The scorecard is a customizable PDF report that developers can easily configure, generate, download, and share with their technical and non-technical stakeholders to educate them about their datasets and models health, achieve compliance, and build trust. This scorecard can also be used in audit reviews to uncover the characteristics of machine learning models.
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-responsible-ai-insights-ui.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ ms.topic: how-to
9
9
ms.reviewer: lagayhar
10
10
ms.author: mithigpe
11
11
author: minthigpen
12
-
ms.date: 11/07/2022
12
+
ms.date: 11/09/2022
13
13
ms.custom: responsible-ml, event-tier1-build-2022
14
14
---
15
15
@@ -25,9 +25,9 @@ To access the dashboard generation wizard and generate a Responsible AI dashboar
25
25
26
26
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/create-responsible-ai-dashboard.png" alt-text="Screenshot of the wizard details pane with 'Create Responsible AI dashboard (preview)' tab highlighted." lightbox ="./media/how-to-responsible-ai-insights-ui/create-responsible-ai-dashboard.png":::
27
27
28
-
To learn more, see the Responsible AI dashboard[supported model types and limitations](concept-responsible-ai-dashboard.md#supported-scenarios-and-limitations).
28
+
To learn more supported model types and limitations in the Responsible AI dashboard, see [supported scenarios and limitations](concept-responsible-ai-dashboard.md#supported-scenarios-and-limitations).
29
29
30
-
The wizard provides an interface for entering all the necessary parameters to create your Responsible AI dashboard without having to touch code. The experience takes place entirely in the Azure Machine Learning studio UI. The studio presents a guided flow and instructional text to help contextualize the variety of choices about which Responsible AI components you’d like to populate your dashboard with.
30
+
The wizard provides an interface for entering all the necessary parameters to create your Responsible AI dashboard without having to touch code. The experience takes place entirely in the Azure Machine Learning studio UI. The studio presents a guided flow and instructional text to help contextualize the variety of choices about which Responsible AI components you’d like to populate your dashboard with.
31
31
32
32
The wizard is divided into five sections:
33
33
@@ -144,7 +144,7 @@ The wizard will allow you to customize your PDF scorecard without having to touc
144
144
1. PDF scorecard summary
145
145
2. Model performance
146
146
3. Tool selection
147
-
4. Data explorer
147
+
4. Data analysis ( previously called data explorer )
148
148
5. Causal analysis
149
149
6. Interpretability
150
150
7. Experiment configuration
@@ -164,17 +164,17 @@ The wizard will allow you to customize your PDF scorecard without having to touc
164
164
165
165
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-selection.png" alt-text="Screenshot of the wizard on scorecard tool selection configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-selection.png":::
166
166
167
-
4.*The Data explorer* section enables cohort analysis. Here, you can identify issues of over- and under-representation explore how data is clustered in the dataset, and how model predictions impact specific data cohorts. Use checkboxes in the dropdown to select your features of interest below to identify your model performance on their underlying cohorts.
167
+
4.*The Data analysis* section enables cohort analysis. Here, you can identify issues of over- and under-representation explore how data is clustered in the dataset, and how model predictions impact specific data cohorts. Use checkboxes in the dropdown to select your features of interest below to identify your model performance on their underlying cohorts.
168
168
169
-
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png" alt-text="Screenshot of the wizard on scorecard data explorer configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png":::
169
+
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png" alt-text="Screenshot of the wizard on scorecard data analysis configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png":::
170
170
171
171
5.*The Fairness assessment* section can help with assessing which groups of people might be negatively impacted by predictions of a machine learning model. There are two fields in this section.
172
172
173
173
- Sensitive features: identify your sensitive attribute(s) of choice (for example, age, gender) by prioritizing up to 20 subgroups you would like to explore and compare.
174
-
174
+
175
175
- Fairness metric: select a fairness metric that is appropriate for your setting (for example, difference in accuracy, error rate ratio), and identify your desired target value(s) on your selected fairness metric(s). Your selected fairness metric (paired with your selection of difference or ratio via the toggle) will capture the difference or ratio between the extreme values across the subgroups. (max - min or max/min).
176
176
177
-
:image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png" alt-text="Screenshot of the wizard on scorecard fairness assessment configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png":::
177
+
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png" alt-text="Screenshot of the wizard on scorecard fairness assessment configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png":::
178
178
179
179
> [!NOTE]
180
180
> The Fairness assessment is currently only available for categorical sensitive attributes such as gender.
@@ -20,7 +20,7 @@ An Azure Machine Learning Responsible AI scorecard is a PDF report that's genera
20
20
21
21
## Where to find your Responsible AI scorecard
22
22
23
-
Responsible AI scorecards are linked to a Responsible AI dashboards. To view your Responsible AI scorecard, go into your model registry by selecting the **Model** in Azure Machine Learning studio. Then select the registered model that you've generated a Responsible AI dashboard and scorecard for. After you've selected your model, select the **Responsible AI** tab to view a list of generated dashboards. Select which dashboard you want to export a Responsible AI scorecard PDF for by selecting **Responsible AI Insights** then **View all PDF scorecards.
23
+
Responsible AI scorecards are linked to your Responsible AI dashboards. To view your Responsible AI scorecard, go into your model registry by selecting the **Model** in Azure Machine Learning studio. Then select the registered model that you've generated a Responsible AI dashboard and scorecard for. After you've selected your model, select the **Responsible AI** tab to view a list of generated dashboards. Select which dashboard you want to export a Responsible AI scorecard PDF for by selecting **Responsible AI Insights** then **View all PDF scorecards.
24
24
25
25
:::image type="content" source="./media/how-to-responsible-ai-scorecard/scorecard-studio.png" alt-text="Screenshot of the 'Responsible AI (preview)' pane in Azure Machine Learning studio, with the 'Responsible AI scorecard (preview)' tab highlighted." lightbox = "./media/how-to-responsible-ai-scorecard/scorecard-studio.png":::
26
26
@@ -64,7 +64,7 @@ Finally, you can see your dataset's causal insights summarized, which can help y
64
64
65
65
## Next steps
66
66
67
-
- See the how-to guide for generating a Responsible AI dashboard via [CLI v2 and SDK v2](how-to-responsible-ai-dashboard-sdk-cli.md) or the [Azure Machine Learning studio UI](how-to-responsible-ai-dashboard-ui.md).
67
+
- See the how-to guide for generating a Responsible AI dashboard via [CLI v2 and SDK v2](how-to-responsible-ai-insights-sdk-cli.md) or the [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
68
68
- Learn more about the [concepts and techniques behind the Responsible AI dashboard](concept-responsible-ai-dashboard.md).
69
69
- View [sample YAML and Python notebooks](https://aka.ms/RAIsamples) to generate a Responsible AI dashboard with YAML or Python.
70
70
- Learn more about how you can use the Responsible AI dashboard and scorecard to debug data and models and inform better decision-making in this [tech community blog post](https://www.microsoft.com/ai/ai-lab-responsible-ai-dashboard).
0 commit comments