Skip to content

Commit 4d6110d

Browse files
committed
addressing reviewer feedback
1 parent 8207fb5 commit 4d6110d

6 files changed

+6
-6
lines changed

articles/machine-learning/how-to-monitor-model-performance.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ ms.custom: devplatv2, update-code
1818

1919
[!INCLUDE [dev v2](includes/machine-learning-dev-v2.md)]
2020

21-
Learn to use Azure Machine Learning's model monitoring to continuously track the performance of machine learning models in production. Model monitoring provides you with a broad view of monitoring signals and alert you to potential issues. When you monitor signals and performance metrics of models in production, you can critically evaluate the inherent risks associated with them and identify blind spots that could adversely affect your business.
21+
Learn to use Azure Machine Learning's model monitoring to continuously track the performance of machine learning models in production. Model monitoring provides you with a broad view of monitoring signals and alerts you to potential issues. When you monitor signals and performance metrics of models in production, you can critically evaluate the inherent risks associated with them and identify blind spots that could adversely affect your business.
2222

2323
In this article you, learn to perform the following tasks:
2424

@@ -406,7 +406,7 @@ To set up advanced monitoring:
406406
1. Select **Next** to open the **Configure data asset** page of the **Advanced settings** section.
407407
1. **Add** a dataset to be used as the reference dataset. We recommend that you use the model training data as the comparison reference dataset for data drift and data quality. Also, use the model validation data as the comparison reference dataset for prediction drift.
408408

409-
:::image type="content" source="media/how-to-monitor-models/model-monitoring-advanced-config-data.png" alt-text="Screenshot showing how to add datasets for the monitoring signals to use." lightbox="media/how-to-monitor-models/model-monitoring-advanced-config-data.png":::
409+
:::image type="content" source="media/how-to-monitor-models/model-monitoring-advanced-configuration-data.png" alt-text="Screenshot showing how to add datasets for the monitoring signals to use." lightbox="media/how-to-monitor-models/model-monitoring-advanced-configuration-data.png":::
410410

411411
1. Select **Next** to go to the **Select monitoring signals** page. On this page, you see some monitoring signals already added (if you selected an Azure Machine Learning online deployment earlier). The signals (data drift, prediction drift, and data quality) use recent, past production data as the comparison reference dataset and use smart defaults for metrics and thresholds.
412412

@@ -449,7 +449,7 @@ To set up advanced monitoring:
449449
1. On the **Notifications** page, enable alert notifications for each signal and select **Next**.
450450
1. Review your settings on the **Review monitoring settings** page.
451451

452-
:::image type="content" source="media/how-to-monitor-models/model-monitoring-advanced-config-review.png" alt-text="Screenshot showing review page of the advanced configuration for model monitoring." lightbox="media/how-to-monitor-models/model-monitoring-advanced-config-review.png":::
452+
:::image type="content" source="media/how-to-monitor-models/model-monitoring-advanced-configuration-review.png" alt-text="Screenshot showing review page of the advanced configuration for model monitoring." lightbox="media/how-to-monitor-models/model-monitoring-advanced-configuration-review.png":::
453453

454454
1. Select **Create** to create your advanced model monitor.
455455

@@ -491,7 +491,7 @@ You must satisfy the following requirements for you to configure your model perf
491491

492492
If you use the [Azure Machine Learning data collector](concept-data-collection.md) to collect production inference data without supplying your own unique ID for each row as a separate column, a `correlationid` will be autogenerated for you and included in the logged JSON object. However, the data collector will [batch rows](how-to-collect-production-data.md#data-collector-batching) that are sent within short time intervals of each other. Batched rows will fall within the same JSON object and will thus have the same `correlationid`.
493493

494-
In order to differentiate between the rows in the same JSON object, Azure Machine Learning model performance monitoring uses indexing to determine the order of the rows in the JSON object. For example, if three rows are batched together, and the `correlationid` is `test`, row one will have an id of `test_0`, row two will have an id of `test_1`, and row three will have an id of `test_2`. To ensure that your ground truth dataset contains unique IDs that match to the collected production inference model outputs, ensure that you index each `correlationid` appropriately. If your logged JSON object only has one row, then the `correlationid` would be `correlationid_0`.
494+
In order to differentiate between the rows in the same JSON object, Azure Machine Learning model performance monitoring uses indexing to determine the order of the rows in the JSON object. For example, if three rows are batched together, and the `correlationid` is `test`, row one will have an ID of `test_0`, row two will have an ID of `test_1`, and row three will have an ID of `test_2`. To ensure that your ground truth dataset contains unique IDs that match to the collected production inference model outputs, ensure that you index each `correlationid` appropriately. If your logged JSON object only has one row, then the `correlationid` would be `correlationid_0`.
495495

496496
To avoid using this indexing, we recommend that you log your unique ID in its own column within the pandas DataFrame that you're logging with the [Azure Machine Learning data collector](how-to-collect-production-data.md). Then, in your model monitoring configuration, you specify the name of this column to join your model output data with your ground truth data. As long as the IDs for each row in both datasets are the same, Azure Machine Learning model monitoring can perform model performance monitoring.
497497

@@ -697,7 +697,7 @@ To set up model performance monitoring:
697697
Ensure that your model outputs dataset is also included in the list of added datasets. The ground truth dataset you add should have a unique ID column.
698698
The values in the unique ID column for both the ground truth dataset and the model outputs dataset must match in order for both datasets to be joined together prior to metric computation.
699699

700-
:::image type="content" source="media/how-to-monitor-models/model-monitoring-advanced-config-data2.png" alt-text="Screenshot showing how to add datasets to use for model performance monitoring." lightbox="media/how-to-monitor-models/model-monitoring-advanced-config-data2.png":::
700+
:::image type="content" source="media/how-to-monitor-models/model-monitoring-advanced-configuration-data-2.png" alt-text="Screenshot showing how to add datasets to use for model performance monitoring." lightbox="media/how-to-monitor-models/model-monitoring-advanced-configuration-data-2.png":::
701701

702702
:::image type="content" source="media/how-to-monitor-models/model-monitoring-added-ground-truth-dataset.png" alt-text="Screenshot showing the ground truth dataset and the model outputs and inputs datasets for the monitoring signals to connect to." lightbox="media/how-to-monitor-models/model-monitoring-added-ground-truth-dataset.png":::
703703

articles/machine-learning/reference-yaml-monitor.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,7 +189,7 @@ Custom monitoring signal through a custom Azure Machine Learning component.
189189
| Key | Type | Description | Allowed values | Default value |
190190
| --- | ---- | ------------| ---------------| ------------- |
191191
| `type` | String | **Required**. Type of monitoring signal. Prebuilt monitoring signal processing component is automatically loaded according to the `type` specified here. | `custom` | `custom` |
192-
| `component_id` | String | **Required**. The Azure Machine Learning component id corresponding to your custom signal. For example `azureml:mycustomcomponent:1` | | |
192+
| `component_id` | String | **Required**. The Azure Machine Learning component ID corresponding to your custom signal. For example `azureml:mycustomcomponent:1` | | |
193193
| `input_data` | Object | **Optional**. Description of the input data to be analyzed by the monitoring signal, see [job input data](./reference-yaml-job-command.md#job-inputs) specification. | | |
194194
| `input_data.<data_name>.data_context` | String | The context of data, it refers model production data and could be model inputs or model outputs | `model_inputs` | |
195195
| `input_data.<data_name>.data_window` | Object | **Optional**. Data window of the reference data to be used as comparison baseline data. | Allow either rolling data window or fixed data window only. For using rolling data window, please specify `input_data.<data_name>.data_window.lookback_window_offset` and `input_data.<data_name>.data_window.lookback_window_size` properties. For using fixed data windows, please specify `input_data.<data_name>.data_window.window_start` and `input_data.<data_name>.data_window.window_end` properties. All property values must be in ISO8601 format. | |

0 commit comments

Comments
 (0)