Skip to content

Commit 806bbe7

Browse files
author
Jill Grant
authored
Merge pull request #264389 from ahughes-msft/momo-doc-model-performance-monitoring
momo - adding section on model performance monitoring
2 parents 0d2a5a5 + 4d6110d commit 806bbe7

12 files changed

+590
-132
lines changed

articles/machine-learning/concept-model-monitoring.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,10 @@ Model monitoring is the last step in the machine learning end-to-end lifecycle.
2424

2525
Unlike traditional software systems, the behavior of machine learning systems is governed not just by rules specified in code, but also by model behavior learned from data. Therefore, data distribution changes, training-serving skew, data quality issues, shifts in environments, or consumer behavior changes can all cause a model to become stale. When a model becomes stale, its performance can degrade to the point that it fails to add business value or starts to cause serious compliance issues in highly regulated environments.
2626

27+
## Limitations of model monitoring in Azure Machine Learning
28+
29+
Azure Machine Learning model monitoring supports only the use of credential-based authentication (e.g., SAS token) to access data contained in datastores. To learn more about datastores and authentication modes, see [Data administration](how-to-administrate-data-authentication.md).
30+
2731
## How model monitoring works in Azure Machine Learning
2832

2933
To implement monitoring, Azure Machine Learning acquires monitoring signals by performing statistical computations on streamed production inference data and reference data. The reference data can be historical training data, validation data, or ground truth data. On the other hand, the production inference data refers to the model's input and output data collected in production.

articles/machine-learning/how-to-collect-production-data.md

Lines changed: 168 additions & 66 deletions
Large diffs are not rendered by default.

articles/machine-learning/how-to-monitor-model-performance.md

Lines changed: 337 additions & 22 deletions
Large diffs are not rendered by default.
69.6 KB
Loading
66.9 KB
Loading
31.5 KB
Loading
56.7 KB
Loading

0 commit comments

Comments
 (0)