Skip to content

Commit bfd5885

Browse files
v-thepetv-thepet
committed
draft
1 parent 5d54915 commit bfd5885

File tree

1 file changed

+68
-64
lines changed

1 file changed

+68
-64
lines changed

‎articles/machine-learning/how-to-responsible-ai-text-insights.md

Lines changed: 68 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -1,140 +1,144 @@
11
---
2-
title: Generate Responsible AI text insights with YAML and Python in Azure Machine Learning
2+
title: Generate Responsible AI text insights with YAML and Python
33
titleSuffix: Azure Machine Learning
4-
description: Learn how to generate Responsible AI text insights with Python and YAML in Azure Machine Learning.
4+
description: Learn how to generate Azure Machine Learning Responsible AI (RAI) text insights by using Python and YAML.
55
services: machine-learning
66
ms.service: machine-learning
77
ms.subservice: rai
88
ms.topic: how-to
99
ms.reviewer: lagayhar
1010
ms.author: wenxwei
1111
author: wenxwei
12-
ms.date: 5/10/2023
12+
ms.date: 5/15/2024
1313
ms.custom: responsible-ml, build-2023, devx-track-python
1414
---
1515

1616
# Generate Responsible AI text insights with YAML and Python (preview)
1717

1818
[!INCLUDE [dev v2](includes/machine-learning-dev-v2.md)]
1919

20-
Understanding and assessing NLP models can be different from tabular data. The Responsible AI dashboard now supports text data by expanding the debugging capabilities and visualizations to be able to digest and visualize text data. The Responsible AI text dashboard provides several mature Responsible AI tools in the areas of error analysis, model interpretability, unfairness assessment and mitigation for a holistic assessment and debugging of NLP models and making informed business decisions. You can generate a Responsible AI text dashboard via a pipeline job by using Responsible AI components.
20+
The [Responsible AI (RAI) dashboard](concept-responsible-ai-dashboard.md) brings together several RAI tools in a single interface to help inform data-driven decisions about your models. Understanding natural language processing (NLP) models can be different from assessing tabular data. RAI dashboard debugging and visualizations now support text data.
2121

22+
The Responsible AI text dashboard provides several mature RAI tools in the areas of error analysis, model interpretability, unfairness assessment, and mitigation. The dashboard supports a holistic assessment and debugging of NLP models for making informed business decisions.
2223

23-
Supported scenarios:
24-
25-
| Name | Description | Parameter name |
26-
|---------------------------------|-----------------------------------------------------|---------------------------------------------|
27-
| Multi-label Text Classification | Predict multiple classes for the given text content | `task_type="multilabel_text_classification` |
24+
This article describes how a pipeline job generates a Responsible AI text dashboard by using the Responsible AI text insights component. The following sections provide specifications and requirements for the text insights component and example code snippets in YAML and Python.
2825

2926
> [!IMPORTANT]
30-
> Responsible AI Text Insights is currently in public preview. This preview is provided without a service-level agreement, and are not recommended for production workloads. Certain features might not be supported or might have constrained capabilities.
31-
> For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
27+
> The Responsible AI text dashboard is currently in public preview. This preview is provided without a service-level agreement, and isn't recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
3228
33-
## Responsible AI component
29+
## Responsible AI text insights component
3430

35-
The core component for constructing the Responsible AI text dashboard in Azure Machine Learning is only the Responsible AI text insights component, which is different from how you construct the Responsible AI pipeline for tabular data.
31+
The Azure Machine Learning Responsible AI text insights component assembles generated insights into a single Responsible AI text dashboard, and is the only core component for constructing the RAI text dashboard. This construction differs from the [Responsible AI pipeline for tabular data](how-to-responsible-ai-insights-sdk-cli.md#responsible-ai-components), which uses several components.
3632

37-
In the following sections are specifications of the Responsible AI text insights component and examples of code snippets in YAML and Python.
33+
### Requirements and limitations
3834

39-
### Limitation
35+
- All models must be registered in Azure Machine Learning.
36+
- MLflow models with a sklearn (scikit-learn) or PyTorch flavor and HuggingFace models are supported.
37+
- The training and test dataset inputs must be in mltable format.
38+
- For performance reason, the test dataset is restricted to 5,000 rows of the visualization UI.
4039

41-
- All models must be registered in Azure Machine Learning.
42-
- Models in MLflow format and with a sklearn or PyTorch flavor are supported.
43-
- HuggingFace models are supported.
44-
- The dataset input must be in mltable format.
45-
- For performance reason, the test dataset is restricted to 5,000 rows of the visualization UI.
40+
### Parameters
4641

47-
## Responsible AI text insights
42+
The Responsible AI text insights `task_type` parameter supports the following scenarios:
43+
44+
| Name | Description | Parameter value |
45+
|---------------------------------|-----------------------------------------------------|---------------------------------------------|
46+
| Text Classification | Predict classes for the given text content | `task_type="text_classification"` |
47+
| Multi-label Text Classification | Predict multiple classes for the given text content | `task_type="multilabel_text_classification"` |
48+
| Text Question Answering | Evaluate a question answering model on the text dataset | `task_type="question_answering"` |
4849

49-
This component has three major input ports:
50+
The Responsible AI text insights component accepts the following other parameters:
51+
52+
| Parameter name | Description | Type |
53+
|----------------|-------------|------|
54+
| `title` | (Optional) Brief description of the dashboard. | String |
55+
| `classes` | (Optional) The full list of class labels in the training dataset. | List of strings |
56+
| `maximum_rows_for_test_dataset` | (Optional) The maximum number of rows allowed in the test dataset, for performance reasons. Defaults to 5,000. | Integer |
57+
| `target_column_name` | The name of the column in the input datasets that the model is trying to predict. | String |
58+
| `enable_explanation` | Enable generating an explanation for the model. | Boolean |
59+
| `enable_error_analysis` | Enable generating an error analysis for the model. | Boolean|
60+
| `use_model_dependency` | The Responsible AI environment doesn't include the model dependencies. When set to `True`, installs the model dependency packages. | Boolean |
61+
| `use_conda` | Installs the model dependency packages using `conda` if set to `True`, otherwise uses `pip`. | Boolean |
62+
63+
### Ports
64+
65+
The Responsible AI text insights component has three major input ports:
5066

5167
- The machine learning model
5268
- The training dataset
5369
- The test dataset
5470

55-
The easiest way to supply the model is to register the input model and reference the same model in the model input port of Responsible AI text insights component.
56-
57-
The two datasets should be in mltable format. The training and test datasets provided don't have to be the same datasets that are used in training the model, but they can be the same.
71+
The easiest way to supply the input model is to register the model and reference the same model in the `model_input` port of the Responsible AI text insights component.
5872

59-
The Responsible AI text insights component also accepts the following parameters:
60-
61-
| Parameter name | Description | Type |
62-
|----------------|-------------|------|
63-
| `title` | Brief description of the dashboard. | String |
64-
| `target_column_name` | The name of the column in the input datasets, which the model is trying to predict. | String |
65-
| `maximum_rows_for_test_dataset` | The maximum number of rows allowed in the test dataset, for performance reasons. | Integer, defaults to 5,000 |
66-
| `classes` | The full list of class labels in the training dataset. | Optional list of strings |
67-
| `enable_explanation` | Enable to generate an explanation for the model. | Boolean |
68-
| `enable_error_analysis` | Enable to generate an error analysis for the model. | Boolean|
69-
| `use_model_dependency` | The Responsible AI environment doesn't include the model dependency, install the model dependency packages when set to True. | Boolean |
70-
| `use_conda` | Install the model dependency packages using conda if True, otherwise using pip. | Boolean |
73+
The training and test datasets don't have to be, but can be the same dataset.
7174

72-
This component assembles the generated insights into a single Responsible AI text dashboard. There are two output ports:
75+
There are two output ports:
7376

74-
- The dashboard port contains the completed RAITextInsights object.
75-
- The ux_json port contains the data required to display a minimal dashboard.
77+
- The `dashboard` port contains the completed `RAITextInsights` object.
78+
- The `ux_json` port contains the data required to display a minimal dashboard.
7679

7780
# [YAML](#tab/yaml)
7881

82+
You can specify the pipeline as a YAML file, as in the following example, and submit it by using the Azure CLI command line.
83+
7984
```yml
85+
jobs:
8086
analyse_model:
8187
type: command
82-
component: azureml://registries/AzureML-RAI-preview/components/rai_text_insights/versions/2
88+
component: azureml://registries/AzureML/components/rai_text_insights/versions/2
8389
inputs:
8490
title: From YAML
8591
task_type: text_classification
8692
model_input:
8793
type: mlflow_model
88-
path: azureml:<registered_model_name>:<registered model version>
89-
model_info: ${{parent.inputs.model_info}}
90-
train_dataset:
91-
type: mltable
92-
path: ${{parent.inputs.my_training_data}}
93-
test_dataset:
94-
type: mltable
95-
path: ${{parent.inputs.my_test_data}}
96-
target_column_name: ${{parent.inputs.target_column_name}}
94+
path: {azureml_model_id}
95+
model_info: ${{{{parent.inputs.model_info}}}}
96+
train_dataset: ${{{{parent.inputs.my_training_data}}}}
97+
test_dataset: ${{{{parent.inputs.my_test_data}}}}
98+
target_column_name: {target_column_name}
9799
maximum_rows_for_test_dataset: 5000
98100
classes: '[]'
99101
enable_explanation: True
100102
enable_error_analysis: True
101-
102103
```
103104
104105
# [Python SDK](#tab/python)
105106
107+
The Responsible AI text dashboard uses the [Responsible AI Text SDK for Python](https://github.com/microsoft/responsible-ai-toolbox/tree/main/responsibleai_text).
108+
109+
First, load the RAI component:
110+
106111
```python
107-
#First load the RAI component:
108112
rai_text_insights_component = ml_client_registry.components.get(
109113
name="rai_text_insights", version=version_string
110-
111114
)
115+
```
116+
117+
Then, inside the pipeline, initiate the RAI Text Insights as in the following example:
112118

113-
#Then inside the pipeline:
114-
# Initiate the RAI Text Insights
119+
```python
115120
rai_text_job = rai_text_insights_component(
116121
title="From Python",
117122
task_type="text_classification",
118123
model_info=expected_model_id,
119-
model_input=Input(type=AssetTypes.MLFLOW_MODEL, path= "<azureml:model_name:model_id>"),
124+
model_input=Input(type=AssetTypes.MLFLOW_MODEL, path= azureml_model_id),
120125
train_dataset=train_data,
121126
test_dataset=test_data,
122127
target_column_name=target_column_name,
123128
classes=classes,
124129
)
130+
```
131+
132+
And assemble the output:
125133

126-
#Assemble the output
134+
```python
127135
rai_text_job.outputs.dashboard.mode = "upload"
128136
rai_text_job.outputs.ux_json.mode = "upload"
129137
```
130138
---
131139

132-
## Understand the Responsible AI text dashboard
133-
134-
To learn more about how to use the Responsible AI text dashboard, see [Responsible AI text dashboard Azure Machine Learning studio](how-to-responsible-ai-text-dashboard.md).
135-
136-
## Next steps
140+
## Related content
137141

142+
- Learn [how to use the Responsible AI text dashboard](how-to-responsible-ai-text-dashboard.md).
138143
- Learn more about the [concepts and techniques behind the Responsible AI dashboard](concept-responsible-ai-dashboard.md).
139-
- View sample [YAML and Python notebooks](https://github.com/Azure/azureml-examples/tree/main/sdk/python/responsible-ai) to generate a Responsible AI dashboard with YAML or Python.
140-
- Learn about how the Responsible AI text dashboard was used by ERM for a [business use case](https://aka.ms/erm-customer-story).
144+
- View sample [YAML and Python notebooks](https://github.com/Azure/azureml-examples/tree/main/sdk/python/responsible-ai) to generate a Responsible AI dashboard.

0 commit comments

Comments
 (0)