You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-responsible-ai-text-dashboard.md
+34-23Lines changed: 34 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -71,8 +71,7 @@ To create a new cohort:
71
71
72
72
1. After you select and configure the filter, select **Add filter**. You can add several filters to define the cohort.
73
73
74
-
1. When you finish configuring and adding filters, select **Save** or **Save and switch** to finalize the new cohort.
75
-
74
+
1. When you're done adding filters, select **Save** or **Save and switch** to finish creating the cohort.
76
75
77
76
## RAI text dashboard components
78
77
@@ -88,17 +87,18 @@ The following sections describe the Responsible AI text dashboard components and
88
87
89
88
### Error analysis
90
89
91
-
The **Error analysis** component helps you analyze failure patterns in your model. The default metric is **Error rate**. You can select a different metric from the **Select metric** dropdown list to learn more about your error and success nodes' performance. This metric selection doesn't impact the way your error tree is generated.
90
+
The **Error analysis** component helps you analyze failure patterns in your model. You can select a metric from the **Select metric** dropdown list to learn more about your error and success nodes' performance. The default metric is **Error rate**. The metric selection doesn't impact the way your error tree is generated.
92
91
93
92
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/error-analysis.png" alt-text="Screenshot of the error analysis component." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/error-analysis.png":::
94
93
95
-
The right side of the **Error analysis** screen shows **Basic Information**about the current cohort and filters applied.
94
+
The right side of the **Error analysis** screen shows information about the dataset, current selected cohort, and filters applied.
96
95
96
+
-**Basic Information** shows the name and filters applied for the current selected cohort.
97
97
-**Instances in global cohort** displays the total number of points and the number of correctly and incorrectly predicted points in the entire dataset.
98
-
-**Instances in the selected cohort** displays the total number of points and the number of correctly and incorrectly predicted points in the current cohort.
99
-
-**Prediction path (filters)** lists the filters applied to the full dataset to create the current cohort.
98
+
-**Instances in the selected cohort** displays the total number of points and the number of correctly and incorrectly predicted points in the currently selected cohort.
99
+
-**Prediction path (filters)** lists the filters applied to the full dataset to create the currently selected cohort.
100
100
101
-
If you change the view by applying filters or selecting different features, you can save the view as a new cohort by selecting **Save as a new cohort**.
101
+
If you change a view by applying filters or selecting different features, you can save the view as a new cohort by selecting **Save as a new cohort**.
102
102
103
103
You can view the error analysis component in **Tree map** or **Heat map** views by selecting either tab at the top of the component.
104
104
@@ -110,12 +110,16 @@ The tree visualization uses the mutual information between each feature and the
110
110
111
111
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/error-analysis-tree.png" alt-text="Screenshot of the tree map view in the error analysis component." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/error-analysis-tree.png":::
112
112
113
-
In the tree view, each *node* represents a dataset cohort, potentially with filters applied, and shows the number of errors in that node compared to the total number of datapoints in the cohort. Hover over or select a node to display the **Error coverage**, or percentage of errors in the dataset that are in the selected node, and **Error rate**, the failure percentage of the datapoints in the selected node.
113
+
In the tree view, each circle or *node* represents a dataset cohort, potentially with filters applied. The numbers in each node show the number of errors compared to the total number of datapoints in that node.
114
+
115
+
You can hover over or select a node to display the **Error coverage**, or percentage of errors in the dataset that are in that node, and **Error rate**, or failure percentage of the datapoints in the node. Nodes with darker background colors indicate higher error rates. Selected nodes have a heavy outline.
114
116
115
-
The *fill lines* between nodes visualize the distribution of datapoints into child cohorts based on filters, with the number of datapoints represented by line thickness. Hover over or select a fill line to display the feature and prediction being applied.
117
+
The *fill lines* between nodes visualize the distribution of datapoints into child cohorts based on filters, with the number of datapoints represented by line thickness. You can hover over or select a fill line to display the feature and prediction being applied.
116
118
117
119
To find important failure patterns, look for nodes with darker colors indicating a higher error rate, and thicker fill lines indicating higher error coverage.
118
120
121
+
##### Feature list
122
+
119
123
To edit the list of features the tree uses to train the tree map, select **Feature list** at the top of the view.
120
124
121
125
In the **Feature list** pane, **Features** lists the names of the features in the dataset. You can search to find specific features. **Importances** visualizes the relative importance of each feature in the dataset. You can select or deselect the check boxes next to each feature to add or remove the feature from the tree map.
@@ -132,13 +136,13 @@ After you make changes in the **Feature list** pane, select **Apply** to apply t
132
136
133
137
#### Heat map view
134
138
135
-
You can use the **Heat map** view to focus on specific intersectional feature filters and compute disaggregated error rates. Start by comparing two dataset features to cluster and filter your data along the two dimensions. You can select one or many heatmap cells and create new cohorts.
139
+
You can use the **Heat map** view to focus on specific intersectional feature filters and compute disaggregated error rates. Start by comparing two dataset features to cluster and filter your data along two dimensions. You can select one or many heatmap cells and create new cohorts.
136
140
137
141
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/error-analysis-heat-map.png" alt-text="Screenshot of the error analysis component in the heat map." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/error-analysis-heat-map.png":::
138
142
139
143
In the **Heat map** view:
140
144
141
-
-**Cells** displays the number of cells selected. If no cells are selected, the view shows all cells with the same data as if all cells were selected.
145
+
-**Cells** displays the number of cells selected. If no cells are selected, the view shows the data for all cells.
142
146
-**Error coverage** displays the percentage of errors in the dataset that are in the selected cells.
143
147
-**Error rate** displays the failure percentage of the datapoints in the selected cells.
144
148
@@ -161,22 +165,29 @@ The **Model overview** component displays model and dataset statistics computed
161
165
162
166
You can choose between **Dataset cohorts** or **Feature cohorts** by selecting either tab at the top of the component. For either view, select **Metric(s)** from the dropdown list or select **Help me choose metrics** to open a sidebar screen that explains, recommends, and lets you select metrics to display.
163
167
164
-
For either view:
168
+
You can also enable **Show heatmaps** for either view.
165
169
166
-
- You can enable **Show heatmaps**.
167
170
- At the bottom of the component, you can choose to view **Metrics visualizations** or a **Confusion matrix**.
168
171
169
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/confusion-matrix.png" alt-text="Screenshot of the top part of the Model overview component." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/confusion-matrix.png":::
172
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/confusion-matrix.png" alt-text="Screenshot of a Confusion matrix in the Feature cohort view of the Model overview component." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/confusion-matrix.png":::
173
+
174
+
#### Dataset cohorts and Feature cohorts
175
+
176
+
**Dataset cohorts** displays the selected metrics across the global cohort and all user-defined cohorts in the dashboard.
177
+
178
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/model-overview.png" alt-text="Screenshot of the model overview on the dataset cohorts tab." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/model-overview.png":::
170
179
171
180
#### Dataset cohorts and Feature cohorts
172
181
173
-
-**Dataset cohorts** displays the selected metrics across the global cohort and all user-defined cohorts in the dashboard.
182
+
**Feature cohorts** displays the selected metrics for cohorts that are based on selected features. You select the **Feature(s)** to define the cohorts. This view also shows **Fairness metrics** such as **Difference** and **Ratio** parity.
183
+
184
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/model-overview-feature-cohort.png" alt-text="Screenshot of the model overview on the feature cohorts tab." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/model-overview-feature-cohort.png":::
174
185
175
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/model-overview.png" alt-text="Screenshot of the model overview on the dataset cohorts tab." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/model-overview.png":::
186
+
#### Metrics visualizations or Confusion matrix
176
187
177
-
-**Feature cohorts** displays the selected metrics for cohorts that are based on selected features. You select the **Feature(s)**to define the cohorts. This view also shows **Fairness metrics** such as **Difference** and **Ratio** parity.
188
+
At the bottom of the component, you can choose to view **Metrics visualizations**or a **Confusion matrix**. The following example shows a confusion matrix for the **Feature cohorts** view.
178
189
179
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/model-overview-feature-cohort.png" alt-text="Screenshot of the model overview on the feature cohorts tab." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/model-overview-feature-cohort.png":::
190
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/confusion-matrix.png" alt-text="Screenshot of a Confusion matrix in the Feature cohort view of the Model overview component." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/confusion-matrix.png":::
180
191
181
192
### Data analysis
182
193
@@ -190,22 +201,22 @@ The **Data analysis** component creates dataset cohorts to analyze dataset stati
190
201
191
202
#### Chart view
192
203
193
-
**Chart view**allows customized aggregate and local data exploration.
204
+
**Chart view**lets you choose between customized aggregation and local data exploration.
194
205
195
206
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/data-analysis-chart-view.png" alt-text="Screenshot of data analysis on the chart view tab." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/data-analysis-chart-view.png":::
196
207
197
-
In the chart view, the X axis and Y axis show the values being plotted horizontally and vertically. You can select either label to open a sidebar pane to select and configure the axis.
208
+
In the chart view, the X axis and Y axis show the values being plotted horizontally and vertically. You can select either label to open a sidebar pane to select and configure that axis.
198
209
199
210
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/axis-value.png" alt-text="Screenshot of the select your axis value sidebar.":::
200
211
201
212
In the **Select your axis value** pane, depending on the value, you can configure options such as **Apply binning to data**, **Enable logarithmic scaling**, or **Treat as categorical**. **Should dither** adds optional noise to the data to avoid overlapping points in the scatterplot.
202
213
203
214
Under **Chart type**, you can select whether to aggregate values across all datapoints.
204
215
205
-
-**Aggregate plot** displays data in bins or categories along the X-axis.
206
-
-**Individual datapoints**shifts to a disaggregated view of the data.
216
+
-**Aggregate plot** displays data in bins or categories along the Xaxis.
217
+
-**Individual datapoints**displays a disaggregated view of the data.
207
218
208
-
In the **Individual datapoints** view, you can select the type of legend used to group datapoints. Selecting the label under **Color value** opens the **Select your axis value** sidebar with the same options that you use to create the chart axes.
219
+
In the **Individual datapoints** view, you can select the type of legend used to group datapoints. Selecting the label under **Color value** opens the **Select your axis value** sidebar pane.
209
220
210
221
:::image type="content" source="./media/how-to-responsible-ai-dashboard-text-insights/data-analysis-chart-individual-datapoints.png" alt-text="Screenshot of data analysis on the chart view tab with individual datapoints option highlighted." lightbox="./media/how-to-responsible-ai-dashboard-text-insights/data-analysis-chart-individual-datapoints.png":::
0 commit comments