Skip to content

Commit 44709a3

Browse files
committed
edits
1 parent be1c026 commit 44709a3

9 files changed

+30
-31
lines changed
Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,29 @@
11
---
22
title: Understand your datasets
33
titleSuffix: Azure Machine Learning
4-
description: Perform exploratory data analysis to understand feature biases and imbalances by using the Responsible AI dashboard's data explorer.
4+
description: Perform exploratory data analysis to understand feature biases and imbalances by using the Responsible AI dashboard's data analysis.
55
services: machine-learning
66
ms.service: machine-learning
77
ms.subservice: enterprise-readiness
88
ms.topic: how-to
99
ms.author: mesameki
1010
author: mesameki
1111
ms.reviewer: lagayhar
12-
ms.date: 08/17/2022
12+
ms.date: 11/09/2022
1313
ms.custom: responsible-ml, event-tier1-build-2022
1414
---
1515

1616
# Understand your datasets (preview)
1717

18-
Machine learning models "learn" from historical decisions and actions captured in training data. As a result, their performance in real-world scenarios is heavily influenced by the data they're trained on. When feature distribution in a dataset is skewed, it can cause a model to incorrectly predict data points that belong to an underrepresented group or to be optimized along an inappropriate metric.
18+
Machine learning models "learn" from historical decisions and actions captured in training data. As a result, their performance in real-world scenarios is heavily influenced by the data they're trained on. When feature distribution in a dataset is skewed, it can cause a model to incorrectly predict data points that belong to an underrepresented group or to be optimized along an inappropriate metric.
1919

2020
For example, while a model was training an AI system for predicting house prices, the training set was representing 75 percent of newer houses that had less than median prices. As a result, it was much less accurate in successfully identifying more expensive historic houses. The fix was to add older and expensive houses to the training data and augment the features to include insights about historical value. That data augmentation improved results.
2121

22-
The data explorer component of the [Responsible AI dashboard](concept-responsible-ai-dashboard.md) helps visualize datasets based on predicted and actual outcomes, error groups, and specific features. It helps you identify issues of overrepresentation and underrepresentation and to see how data is clustered in the dataset. Data visualizations consist of aggregate plots or individual data points.
22+
The data analysis component of the [Responsible AI dashboard](concept-responsible-ai-dashboard.md) helps visualize datasets based on predicted and actual outcomes, error groups, and specific features. It helps you identify issues of overrepresentation and underrepresentation and to see how data is clustered in the dataset. Data visualizations consist of aggregate plots or individual data points.
2323

24-
## When to use the data explorer
24+
## When to use data analysis
2525

26-
Use the data explorer when you need to:
26+
Use data analysis when you need to:
2727

2828
- Explore your dataset statistics by selecting different filters to slice your data into different dimensions (also known as cohorts).
2929
- Understand the distribution of your dataset across different cohorts and feature groups.
@@ -33,5 +33,5 @@ Use the data explorer when you need to:
3333
## Next steps
3434

3535
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-insights-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
36-
- Explore the [supported data explorer visualizations](how-to-responsible-ai-dashboard.md#data-analysis) of the Responsible AI dashboard.
36+
- Explore the [supported data analysis visualizations](how-to-responsible-ai-dashboard.md#data-analysis) of the Responsible AI dashboard.
3737
- Learn how to generate a [Responsible AI scorecard](how-to-responsible-ai-scorecard.md) based on the insights observed in the Responsible AI dashboard.

articles/machine-learning/concept-fairness-ml.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ The Fairlearn open-source package provides two types of unfairness mitigation al
9090

9191
## Next steps
9292

93-
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-dashboard-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-dashboard-ui.md).
93+
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-insights-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
9494
- Explore the [supported model overview and fairness assessment visualizations](how-to-responsible-ai-dashboard.md#model-overview-and-fairness-metrics) of the Responsible AI dashboard.
9595
- Learn how to generate a [Responsible AI scorecard](how-to-responsible-ai-scorecard.md) based on the insights observed in the Responsible AI dashboard.
9696
- Learn how to use the components by checking out Fairlearn's [GitHub repository](https://github.com/fairlearn/fairlearn/), [user guide](https://fairlearn.github.io/main/user_guide/index.html), [examples](https://fairlearn.github.io/main/auto_examples/index.html), and [sample notebooks](https://github.com/fairlearn/fairlearn/tree/master/notebooks).

articles/machine-learning/concept-responsible-ai-dashboard.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: how-to
99
ms.author: mesameki
1010
author: mesameki
1111
ms.reviewer: lagayhar
12-
ms.date: 11/07/2022
12+
ms.date: 11/09/2022
1313
ms.custom: responsible-ml, event-tier1-build-2022
1414
---
1515

@@ -174,5 +174,5 @@ The following people can use the Responsible AI dashboard, and its corresponding
174174

175175
## Next steps
176176

177-
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-dashboard-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-dashboard-ui.md).
177+
- Learn how to generate the Responsible AI dashboard via [CLI and SDK](how-to-responsible-ai-insights-sdk-cli.md) or [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
178178
- Learn how to generate a [Responsible AI scorecard](concept-responsible-ai-scorecard.md) based on the insights observed on the Responsible AI dashboard.

articles/machine-learning/concept-responsible-ai-scorecard.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: conceptual
99
ms.author: mesameki
1010
author: mesameki
1111
ms.reviewer: lagayhar
12-
ms.date: 11/07/2022
12+
ms.date: 11/09/2022
1313
ms.custom: responsible-ml
1414
---
1515

articles/machine-learning/concept-responsible-ai.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: conceptual
99
ms.author: mesameki
1010
author: mesameki
1111
ms.reviewer: lagayhar
12-
ms.date: 11/07/2022
12+
ms.date: 11/09/2022
1313
ms.custom: responsible-ai, event-tier1-build-2022
1414
#Customer intent: As a data scientist, I want to learn what Responsible AI is and how I can use it in Azure Machine Learning.
1515
---
@@ -57,9 +57,9 @@ The model interpretability component provides multiple views into a model's beha
5757

5858
- *Global explanations*. For example, what features affect the overall behavior of a loan allocation model?
5959
- *Local explanations*. For example, why was a customer's loan application approved or rejected?
60-
- *Model explanations for a selected cohort of data points*. For example, what features affect the overall behavior of a loan allocation model for low-income applicants?
60+
- *Model explanations for a selected cohort of data points*. For example, what features affect the overall behavior of a loan allocation model for low-income applicants?
6161

62-
The counterfactual what-if component enables understanding and debugging a machine learning model in terms of how it reacts to feature changes and perturbations.
62+
The counterfactual what-if component enables understanding and debugging a machine learning model in terms of how it reacts to feature changes and perturbations.
6363

6464
Azure Machine Learning also supports a [Responsible AI scorecard](./how-to-responsible-ai-scorecard.md). The scorecard is a customizable PDF report that developers can easily configure, generate, download, and share with their technical and non-technical stakeholders to educate them about their datasets and models health, achieve compliance, and build trust. This scorecard can also be used in audit reviews to uncover the characteristics of machine learning models.
6565

articles/machine-learning/how-to-responsible-ai-dashboard.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: how-to
99
ms.reviewer: lagayhar
1010
ms.author: mithigpe
1111
author: minthigpen
12-
ms.date: 11/07/2022
12+
ms.date: 11/09/2022
1313
ms.custom: responsible-ml, event-tier1-build-2022
1414
---
1515

articles/machine-learning/how-to-responsible-ai-insights-sdk-cli.md

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: how-to
99
ms.reviewer: lagayhar
1010
ms.author: mithigpe
1111
author: minthigpen
12-
ms.date: 11/07/2022
12+
ms.date: 11/09/2022
1313
ms.custom: responsible-ml, event-tier1-build-2022
1414
---
1515

@@ -275,7 +275,7 @@ This component has a single output port, which can be connected to one of the `i
275275
```yml
276276
  explain_01:
277277
    type: command
278-
    component: azureml:rai_insights_explanation:VERSION_REPLACEMENT_STRING
278+
    component: azureml://registries/azureml/components/microsoft_azureml_rai_tabular_explanation/versions/<version>
279279
    inputs:
280280
      comment: My comment
281281
      rai_insights_dashboard: ${{parent.jobs.create_rai_job.outputs.rai_insights_dashboard}}
@@ -286,9 +286,8 @@ This component has a single output port, which can be connected to one of the `i
286286

287287
```python
288288
#First load the component:
289-
        rai_explanation_component = load_component(
290-
            client=ml_client, name="rai_insights_explanation", version="1"
291-
        ) 1
289+
        rai_explanation_component = ml_client_registry.components.get(name="microsoft_azureml_rai_tabular_explanation", label="latest"
290+
292291
#Use inside a pipeline:
293292
            explain_job = rai_explanation_component(
294293
                comment="My comment",

articles/machine-learning/how-to-responsible-ai-insights-ui.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ ms.topic: how-to
99
ms.reviewer: lagayhar
1010
ms.author: mithigpe
1111
author: minthigpen
12-
ms.date: 11/07/2022
12+
ms.date: 11/09/2022
1313
ms.custom: responsible-ml, event-tier1-build-2022
1414
---
1515

@@ -25,9 +25,9 @@ To access the dashboard generation wizard and generate a Responsible AI dashboar
2525

2626
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/create-responsible-ai-dashboard.png" alt-text="Screenshot of the wizard details pane with 'Create Responsible AI dashboard (preview)' tab highlighted." lightbox ="./media/how-to-responsible-ai-insights-ui/create-responsible-ai-dashboard.png":::
2727

28-
To learn more, see the Responsible AI dashboard [supported model types and limitations](concept-responsible-ai-dashboard.md#supported-scenarios-and-limitations).
28+
To learn more supported model types and limitations in the Responsible AI dashboard, see [supported scenarios and limitations](concept-responsible-ai-dashboard.md#supported-scenarios-and-limitations).
2929

30-
The wizard provides an interface for entering all the necessary parameters to create your Responsible AI dashboard without having to touch code. The experience takes place entirely in the Azure Machine Learning studio UI. The studio presents a guided flow and instructional text to help contextualize the variety of choices about which Responsible AI components you’d like to populate your dashboard with.
30+
The wizard provides an interface for entering all the necessary parameters to create your Responsible AI dashboard without having to touch code. The experience takes place entirely in the Azure Machine Learning studio UI. The studio presents a guided flow and instructional text to help contextualize the variety of choices about which Responsible AI components you’d like to populate your dashboard with.
3131

3232
The wizard is divided into five sections:
3333

@@ -144,7 +144,7 @@ The wizard will allow you to customize your PDF scorecard without having to touc
144144
1. PDF scorecard summary
145145
2. Model performance
146146
3. Tool selection
147-
4. Data explorer
147+
4. Data analysis ( previously called data explorer )
148148
5. Causal analysis
149149
6. Interpretability
150150
7. Experiment configuration
@@ -164,17 +164,17 @@ The wizard will allow you to customize your PDF scorecard without having to touc
164164

165165
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-selection.png" alt-text="Screenshot of the wizard on scorecard tool selection configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-selection.png":::
166166

167-
4. *The Data explorer* section enables cohort analysis. Here, you can identify issues of over- and under-representation explore how data is clustered in the dataset, and how model predictions impact specific data cohorts. Use checkboxes in the dropdown to select your features of interest below to identify your model performance on their underlying cohorts.
167+
4. *The Data analysis* section enables cohort analysis. Here, you can identify issues of over- and under-representation explore how data is clustered in the dataset, and how model predictions impact specific data cohorts. Use checkboxes in the dropdown to select your features of interest below to identify your model performance on their underlying cohorts.
168168

169-
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png" alt-text="Screenshot of the wizard on scorecard data explorer configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png":::
169+
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png" alt-text="Screenshot of the wizard on scorecard data analysis configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-explorer.png":::
170170

171171
5. *The Fairness assessment* section can help with assessing which groups of people might be negatively impacted by predictions of a machine learning model. There are two fields in this section.
172172

173173
- Sensitive features: identify your sensitive attribute(s) of choice (for example, age, gender) by prioritizing up to 20 subgroups you would like to explore and compare.
174-
174+
175175
- Fairness metric: select a fairness metric that is appropriate for your setting (for example, difference in accuracy, error rate ratio), and identify your desired target value(s) on your selected fairness metric(s). Your selected fairness metric (paired with your selection of difference or ratio via the toggle) will capture the difference or ratio between the extreme values across the subgroups. (max - min or max/min).
176176

177-
:image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png" alt-text="Screenshot of the wizard on scorecard fairness assessment configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png":::
177+
:::image type="content" source="./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png" alt-text="Screenshot of the wizard on scorecard fairness assessment configuration." lightbox= "./media/how-to-responsible-ai-insights-ui/scorecard-fairness.png":::
178178

179179
> [!NOTE]
180180
> The Fairness assessment is currently only available for categorical sensitive attributes such as gender.

articles/machine-learning/how-to-responsible-ai-scorecard.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ ms.subservice: enterprise-readiness
88
ms.topic: how-to
99
ms.author: mesameki
1010
author: mesameki
11-
ms.date: 11/07/2022
11+
ms.date: 11/09/2022
1212
ms.custom: responsible-ml, event-tier1-build-2022
1313
---
1414

@@ -20,7 +20,7 @@ An Azure Machine Learning Responsible AI scorecard is a PDF report that's genera
2020

2121
## Where to find your Responsible AI scorecard
2222

23-
Responsible AI scorecards are linked to a Responsible AI dashboards. To view your Responsible AI scorecard, go into your model registry by selecting the **Model** in Azure Machine Learning studio. Then select the registered model that you've generated a Responsible AI dashboard and scorecard for. After you've selected your model, select the **Responsible AI** tab to view a list of generated dashboards. Select which dashboard you want to export a Responsible AI scorecard PDF for by selecting **Responsible AI Insights** then **View all PDF scorecards.
23+
Responsible AI scorecards are linked to your Responsible AI dashboards. To view your Responsible AI scorecard, go into your model registry by selecting the **Model** in Azure Machine Learning studio. Then select the registered model that you've generated a Responsible AI dashboard and scorecard for. After you've selected your model, select the **Responsible AI** tab to view a list of generated dashboards. Select which dashboard you want to export a Responsible AI scorecard PDF for by selecting **Responsible AI Insights** then **View all PDF scorecards.
2424

2525
:::image type="content" source="./media/how-to-responsible-ai-scorecard/scorecard-studio.png" alt-text="Screenshot of the 'Responsible AI (preview)' pane in Azure Machine Learning studio, with the 'Responsible AI scorecard (preview)' tab highlighted." lightbox = "./media/how-to-responsible-ai-scorecard/scorecard-studio.png":::
2626

@@ -64,7 +64,7 @@ Finally, you can see your dataset's causal insights summarized, which can help y
6464

6565
## Next steps
6666

67-
- See the how-to guide for generating a Responsible AI dashboard via [CLI&nbsp;v2 and SDK&nbsp;v2](how-to-responsible-ai-dashboard-sdk-cli.md) or the [Azure Machine Learning studio UI](how-to-responsible-ai-dashboard-ui.md).
67+
- See the how-to guide for generating a Responsible AI dashboard via [CLI&nbsp;v2 and SDK&nbsp;v2](how-to-responsible-ai-insights-sdk-cli.md) or the [Azure Machine Learning studio UI](how-to-responsible-ai-insights-ui.md).
6868
- Learn more about the [concepts and techniques behind the Responsible AI dashboard](concept-responsible-ai-dashboard.md).
6969
- View [sample YAML and Python notebooks](https://aka.ms/RAIsamples) to generate a Responsible AI dashboard with YAML or Python.
7070
- Learn more about how you can use the Responsible AI dashboard and scorecard to debug data and models and inform better decision-making in this [tech community blog post](https://www.microsoft.com/ai/ai-lab-responsible-ai-dashboard).

0 commit comments

Comments
 (0)