You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Use the responsible AI image dashboard (preview)
17
17
18
-
[Responsible AI (RAI) dashboards](how-to-responsible-ai-dashboard.md) include a robust, rich set of visualizations and functionality to help you analyze your machine learning model or make data-driven business decisions. The steps to view and configure RAI dashboards are similar across scenarios, but some features are unique to image scenarios. This article describes how to access and configure RAI image dashboard components and functionality.
18
+
[Responsible AI (RAI) dashboards](how-to-responsible-ai-dashboard.md) include visualizations and functionality to help you analyze your machine learning model and make data-driven business decisions. The steps to view and configure RAI dashboards are similar across scenarios, but some features are unique to image scenarios. This article describes how to access and configure RAI image dashboard components and functionality.
19
19
20
20
Responsible AI image dashboards are linked to your registered computer vision models in Azure Machine Learning. The [Responsible AI Vision Insights component](how-to-responsible-ai-vision-insights.md) supports image classification and object detection scenarios.
21
21
@@ -80,11 +80,11 @@ The **Vision data explorer** component has several views to provide various pers
80
80
81
81
The **Image explorer** view displays image instances of model predictions, automatically categorized by correctly and incorrectly labeled predictions. This view helps you quickly identify high-level error patterns in your data and select which instances to investigate further.
82
82
83
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/image-explorer.png" alt-text="Screenshot of image explorer for multilabel classification." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/image-explorer.png":::
84
-
85
-
- Under **Select a dataset cohort to explore**, you can select or search for a user-defined dataset cohort to view.
83
+
-**Select a dataset cohort to explore** lets you select or search for a user-defined dataset cohort to view.
86
84
- The **Set thumbnail size** slider adjusts the size of the image cards on the page.
87
85
86
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/image-explorer.png" alt-text="Screenshot of image explorer view in Vision data explorer." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/image-explorer.png":::
87
+
88
88
# [Image classification](#tab/classification)
89
89
90
90
For image classification and multiclassification, incorrect predictions refer to images where the predicted class label differs from ground truth. Each image card displays the image, the predicted class label, and the ground truth class label.
@@ -113,31 +113,31 @@ Each image card displays the image and how many object predictions were **Correc
113
113
114
114
Selecting an image card opens a flyout to view components that support the analysis of model predictions.
115
115
116
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object-iou.png" alt-text="Screenshot of selected instance with bounding boxes and IOU score." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object-iou.png":::
117
+
116
118
-**Selected instance**: The correct detections and incorrect detections for the image or objects in the image.
117
119
-**Information**: Image metadata values for the selected instance.
118
120
- An enlarged image with predicted and ground truth objects surrounded by bounding boxes, and the predicted image labeled with its IOU score.
119
121
120
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object-iou.png" alt-text="Screenshot of selected instance with bounding boxes and IOU score." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object-iou.png":::
121
-
122
-
> [!NOTE]
123
-
> If an object in an image was correctly labeled, but with an IOU score below the default threshold of 50%, the prediction bounding box for the object isn't visible, but the ground truth bounding box is visible. The image instance appears in the error instance category. Currently, it's not possible to change the default IOU threshold in this component.
122
+
> [!NOTE]
123
+
> If an object in an image was correctly labeled, but with an IOU score below the default threshold of 50%, the prediction bounding box for the object isn't visible, but the ground truth bounding box is visible. The image instance appears in the error instance category. Currently, it's not possible to change the default IOU threshold in this component.
124
124
125
125
-**Explanation**: A D-Rise saliency map visualization to gain insight on model behavior leading to the execution of the computer vision task.
126
126
127
-
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object.png" alt-text="Screenshot of selected instance with an image of a can and the explanation for object detection." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object.png":::
127
+
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object.png" alt-text="Screenshot of selected instance with an image of a can and the explanation for object detection." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/image-instance-object.png":::
128
128
129
129
---
130
130
131
131
#### Table view
132
132
133
-
The **Table view** shows your dataset with rows for each image instance and columns for the corresponding index, ground truth class labels, predicted class labels, and metadata features.
133
+
**Table view** shows your dataset with rows for each image instance and columns for the corresponding index, ground truth class labels, predicted class labels, and metadata features.
134
134
135
135
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/table-view-common.png" alt-text="Screenshot of the top part of the Table view." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/table-view-common.png":::
136
136
137
137
-**Select a dataset cohort to explore** lets you can select or search for a user-defined dataset cohort to view.
138
-
-**Save cohort** lets you create new dataset cohorts by manually selecting images.
138
+
-**Save cohort** lets you save new dataset cohorts you create by manually selecting images.
139
139
140
-
Hover on an image row and select the checkbox to include the image in the new cohort. You can filter your dataset by index, metadata values, and classification outcome. You can add more filters, save the resulting filtered data with a new cohort name, and automatically switch to display your new cohort.
140
+
Hover on an image row and select the checkbox to include the image in the new cohort. You can filter your dataset by index, metadata values, and classification outcome. You can add more filters, save the resulting filtered data with a new cohort name, and automatically switch to display the new cohort.
141
141
142
142
# [Image classification](#tab/classification)
143
143
@@ -198,8 +198,6 @@ You can also choose to **Show heatmap** for the displayed data.
198
198
199
199
Recommended metrics for image classification and image multilabel classification include **Accuracy score**, **Macro Precision score**, **Macro Recall score**, and **Macro F1 score**.
200
200
201
-
This view also shows **Fairness metrics** such as **Difference** and **Ratio** parity.
202
-
203
201
:::image type="content" source="./media/how-to-responsible-ai-dashboard-vision-insights/dataset-cohorts.png" alt-text="Screenshot of model overview on the dataset cohorts pane." lightbox="./media/how-to-responsible-ai-dashboard-vision-insights/dataset-cohorts.png":::
204
202
205
203
# [Object detection](#tab/detection)
@@ -210,7 +208,7 @@ For object detection, in addition to selecting the **Metric(s)**, configure the
210
208
211
209
-**Aggregate method**: Select which aggregation method to apply, which affects the calculation of mean average precision.
212
210
-**Select class(es)**: Select which class label to use to calculate class-level metrics like average precision and average recall.
213
-
-**IoU Threshold**: Set an IOU threshold value for the Intersection of Union between the ground truth and prediction bounding box. The IOU threshold value defines error and affects calculation of model performance metrics.
211
+
-**IoU Threshold**: Set an IOU threshold value for the Intersection over Union between the ground truth and prediction bounding box. The IOU threshold value defines error and affects calculation of model performance metrics.
214
212
215
213
For example, setting an IOU of greater than 70% means that a prediction with greater than 70% overlap with ground truth is `True`. This feature is disabled by default, and can be enabled by attaching a running compute instance.
216
214
@@ -220,7 +218,9 @@ For object detection, in addition to selecting the **Metric(s)**, configure the
220
218
221
219
#### Feature cohorts view
222
220
223
-
In the **Feature cohorts** view, you can investigate your model by comparing model performance across user-specified sensitive and nonsensitive features. For example, compare performance for cohorts across various image metadata values like gender, race, and income. To learn more about feature cohorts, see [Feature cohorts](how-to-responsible-ai-dashboard.md#feature-cohorts).
221
+
In the **Feature cohorts** view, you can investigate your model by comparing model performance across user-specified sensitive and nonsensitive features. For example, compare performance for cohorts across various image metadata values like gender, race, and income.
222
+
223
+
This view also shows **Fairness metrics** such as **Difference** and **Ratio** parity. To learn more about feature cohorts, see [Feature cohorts](how-to-responsible-ai-dashboard.md#feature-cohorts).
224
224
225
225
# [Image classification](#tab/classification)
226
226
@@ -236,7 +236,7 @@ In the **Feature cohorts** view, you can investigate your model by comparing mod
236
236
237
237
# [Image classification](#tab/classification)
238
238
239
-
In the lower half of both the **Dataset cohorts**and **Feature cohorts** views, you can select between the following visualizations of the data in that view:
239
+
In the lower half of the **Model overview**component, you can select between **Metrics visualizations** or a **Confusion matrix**view of the data:
240
240
241
241
-**Metrics visualizations** shows a bar graph that compares aggregated performance metrics across selected dataset cohorts.
242
242
- Select **Choose cohorts** to open a sidebar that lets you select the dataset and feature cohorts to apply.
0 commit comments