Merge pull request #49853 from lootle1/MR86

Technical Review 1037456: Measure and optimize model performance with…

Author: v-ccolin

learn-pr/azure/optimize-model-performance-roc-auc/1-introduction.yml

@@ -1,13 +1,13 @@
-### YamlMime:ModuleUnit
-uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.introduction
-title: Introduction
-metadata:
-  title: Introduction
-  description: Introduction to the ROC AUC module.
-  ms.date: 07/20/2024
-  author: s-polly
-  ms.author: scottpolly
-  ms.topic: unit
-durationInMinutes: 2
-content: |
-  [!include[](includes/1-introduction.md)] 
+### YamlMime:ModuleUnit
+uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.introduction
+title: Introduction
+metadata:
+  title: Introduction
+  description: Introduction to the ROC AUC module.
+  ms.date: 04/03/2025
+  author: s-polly
+  ms.author: scottpolly
+  ms.topic: unit
+durationInMinutes: 2
+content: |
+  [!include[](includes/1-introduction.md)] 

learn-pr/azure/optimize-model-performance-roc-auc/2-receiver-operator-characteristic-curve.yml

@@ -1,13 +1,13 @@
-### YamlMime:ModuleUnit
-uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.receiver-operator-characteristic-curve
-title: Analyze classification with receiver operator characteristic curves
-metadata:
-  title: Analyze classification with receiver operator characteristic curves
-  description: Conceptual unit introducing ROC curves in machine learning
-  ms.date: 07/20/2024
-  author: s-polly
-  ms.author: scottpolly
-  ms.topic: unit
-durationInMinutes: 4
-content: |
-  [!include[](includes/2-receiver-operator-characteristic-curve.md)]
+### YamlMime:ModuleUnit
+uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.receiver-operator-characteristic-curve
+title: Analyze classification with receiver operator characteristic curves
+metadata:
+  title: Analyze Classification with Receiver Operator Characteristic Curves
+  description: Conceptual unit introducing ROC curves in machine learning
+  ms.date: 04/03/2025
+  author: s-polly
+  ms.author: scottpolly
+  ms.topic: unit
+durationInMinutes: 4
+content: |
+  [!include[](includes/2-receiver-operator-characteristic-curve.md)]

learn-pr/azure/optimize-model-performance-roc-auc/3-exercise-evaluate-roc-curves.yml

@@ -1,14 +1,14 @@
-### YamlMime:ModuleUnit
-uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.exercise-evaluate-roc-curves
-title: Exercise - Evaluate ROC curves
-metadata:
-  title: Exercise - Evaluate ROC curves
-  description: Exercise about good and bad ROC curves in machine learning
-  ms.date: 07/20/2024
-  author: s-polly
-  ms.author: scottpolly
-  ms.topic: unit
-durationInMinutes: 8
-sandbox: true
-notebook: notebooks/9-3-evaluate-roc-curves.ipynb
-
+### YamlMime:ModuleUnit
+uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.exercise-evaluate-roc-curves
+title: Exercise - Evaluate ROC curves
+metadata:
+  title: Exercise - Evaluate ROC Curves
+  description: Exercise about good and bad ROC curves in machine learning
+  ms.date: 04/03/2025
+  author: s-polly
+  ms.author: scottpolly
+  ms.topic: unit
+durationInMinutes: 8
+sandbox: true
+notebook: notebooks/9-3-evaluate-roc-curves.ipynb
+

learn-pr/azure/optimize-model-performance-roc-auc/4-compare-optimize-curves.yml

@@ -1,14 +1,14 @@
-### YamlMime:ModuleUnit
-uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.comparing-optimizing-curves
-title: Compare and optimize ROC curves
-metadata:
-  title: Compare and optimize ROC curves
-  description: Conceptual unit about comparing and optimizing machine learning models ROC curves
-  ms.date: 07/20/2024
-  author: s-polly
-  ms.author: scottpolly
-  ms.topic: unit
-durationInMinutes: 4
-content: |
-  [!include[](includes/4-compare-optimize-curves.md)]
-
+### YamlMime:ModuleUnit
+uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.comparing-optimizing-curves
+title: Compare and optimize ROC curves
+metadata:
+  title: Compare and Optimize ROC Curves
+  description: Conceptual unit about comparing and optimizing machine learning models ROC curves
+  ms.date: 04/03/2025
+  author: s-polly
+  ms.author: scottpolly
+  ms.topic: unit
+durationInMinutes: 4
+content: |
+  [!include[](includes/4-compare-optimize-curves.md)]
+

learn-pr/azure/optimize-model-performance-roc-auc/5-tune-auc-curves.yml

@@ -1,14 +1,14 @@
-### YamlMime:ModuleUnit
-uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.exercise-tune-auc-curves
-title: Exercise - Tune the area under the curve
-metadata:
-  title: Exercise - Tune the area under the curve
-  description: Exercise unit about tuning under the curve in machine learning
-  ms.date: 07/20/2024
-  author: s-polly
-  ms.author: scottpolly
-  ms.topic: unit
-durationInMinutes: 12
-sandbox: true
-notebook: notebooks/9-5-tune-auc-curves.ipynb
-
+### YamlMime:ModuleUnit
+uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.exercise-tune-auc-curves
+title: Exercise - Tune the area under the curve
+metadata:
+  title: Exercise - Tune the Area Under the Curve
+  description: Exercise unit about tuning under the curve in machine learning
+  ms.date: 04/03/2025
+  author: s-polly
+  ms.author: scottpolly
+  ms.topic: unit
+durationInMinutes: 12
+sandbox: true
+notebook: notebooks/9-5-tune-auc-curves.ipynb
+

learn-pr/azure/optimize-model-performance-roc-auc/8-knowledge-check.yml

@@ -1,48 +1,48 @@
-### YamlMime:ModuleUnit
-uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.knowledge-check
-title: Module assessment
-metadata:
-  title: Module assessment
-  description: Multiple-choice questions
-  ms.date: 07/20/2024
-  author: s-polly
-  ms.author: scottpolly
-  ms.topic: unit
-durationInMinutes: 3
-quiz:
-  title: Check your knowledge
-  questions:
-  - content: 'What do TPR and FPR mean?'
-    choices:
-    - content: "TPR is the number of correct responses. FPR is the number of incorrect responses."
-      isCorrect: false
-      explanation: "Incorrect."
-    - content: "TPR is the proportion of answers that were provided correctly as 'true'. FPR is the proportion of answers that were provided incorrectly as 'true'."
-      isCorrect: true
-      explanation: "Correct."
-    - content: "TPR is the proportion of answers that were provided correctly as 'true'. FPR is the proportion of answers that were provided incorrectly as 'false'."
-      isCorrect: false
-      explanation: "Incorrect."
-  - content: 'What are on the X and Y axes in an ROC plot?'
-    choices:
-    - content: 'X-axis: FP rate, Y-axis: TP rate'
-      isCorrect: true
-      explanation: "Correct."
-    - content: "X-axis: Number of FPs, Y-axis: Number of TPs"
-      isCorrect: false
-      explanation: "Incorrect. "
-    - content: "X-axis: Number of TPs, Y-axis: Number of FPs"
-      isCorrect: false
-      explanation: "Incorrect."
-  - content: 'What does area under the curve for an ROC plot tell us?'
-    choices:
-    - content: "How well the model works at its optimum decision threshold"
-      isCorrect: false
-      explanation: "Incorrect. This information might be obtainable from the ROC plot but we can't get this information from the AUC."
-    - content: "Which is the optimum decision threshold?"
-      isCorrect: false
-      explanation: "Incorrect. AUC is a summary metric that is too simplified to provide this information."
-    - content: "It gives a summary of how well a model works across various thresholds."
-      isCorrect: true
-      explanation: "Correct."
-
+### YamlMime:ModuleUnit
+uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.knowledge-check
+title: Module Assessment
+metadata:
+  title: Module Assessment
+  description: Multiple-choice questions
+  ms.date: 04/03/2025
+  author: s-polly
+  ms.author: scottpolly
+  ms.topic: unit
+durationInMinutes: 3
+quiz:
+  title: Check your knowledge
+  questions:
+  - content: 'What do TPR and FPR mean?'
+    choices:
+    - content: "TPR is the number of correct responses. FPR is the number of incorrect responses."
+      isCorrect: false
+      explanation: "Incorrect."
+    - content: "TPR is the proportion of answers that were provided correctly as 'true.' FPR is the proportion of answers that were provided incorrectly as 'true.'"
+      isCorrect: true
+      explanation: "Correct."
+    - content: "TPR is the proportion of answers that were provided correctly as 'true.' FPR is the proportion of answers that were provided incorrectly as 'false.'"
+      isCorrect: false
+      explanation: "Incorrect."
+  - content: 'What are on the X and Y axes in an ROC plot?'
+    choices:
+    - content: 'X-axis: FP rate, Y-axis: TP rate'
+      isCorrect: true
+      explanation: "Correct."
+    - content: "X-axis: Number of FPs, Y-axis: Number of TPs"
+      isCorrect: false
+      explanation: "Incorrect. "
+    - content: "X-axis: Number of TPs, Y-axis: Number of FPs"
+      isCorrect: false
+      explanation: "Incorrect."
+  - content: 'What does area under the curve for an ROC plot tell us?'
+    choices:
+    - content: "How well the model works at its optimum decision threshold"
+      isCorrect: false
+      explanation: "Incorrect. This information might be obtainable from the ROC plot, but we can't get this information from the AUC."
+    - content: "Which is the optimum decision threshold?"
+      isCorrect: false
+      explanation: "Incorrect. AUC is a summary metric that is too simplified to provide this information."
+    - content: "It gives a summary of how well a model works across various thresholds."
+      isCorrect: true
+      explanation: "Correct."
+

learn-pr/azure/optimize-model-performance-roc-auc/9-summary.yml

@@ -1,13 +1,13 @@
-### YamlMime:ModuleUnit
-uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.summary
-title: Summary
-metadata:
-  title: Summary
-  description: An overview of the content covered in the module.
-  ms.date: 07/20/2024
-  author: s-polly
-  ms.author: scottpolly
-  ms.topic: unit
-durationInMinutes: 3
-content: |
-  [!include[](includes/9-summary.md)] 
+### YamlMime:ModuleUnit
+uid: learn.machinelearning.optimize-model-performance-roc-auc-dropout.summary
+title: Summary
+metadata:
+  title: Summary
+  description: An overview of the content covered in the module.
+  ms.date: 04/03/2025
+  author: s-polly
+  ms.author: scottpolly
+  ms.topic: unit
+durationInMinutes: 3
+content: |
+  [!include[](includes/9-summary.md)] 

learn-pr/azure/optimize-model-performance-roc-auc/includes/1-introduction.md

@@ -1,8 +1,8 @@
-We can assess our classification models in terms of the kinds of mistakes that they make, such as false negatives and false positives. This can give insight into the kinds of mistakes a model makes, but doesn't necessarily provide deep information on how the model could perform if slight adjustments were made to its decision criteria. Here, we'll discuss receiver operator characteristic (ROC) curves, which build on the idea of a confusion matrix but provide us with deeper information that lets us improve our models to a greater degree.
+We can assess our classification models in terms of the kinds of mistakes that they make, such as false negatives and false positives. This can give insight into the kinds of mistakes a model makes, but it doesn't necessarily provide deep information on how the model could perform if slight adjustments were made to its decision criteria. Here, we'll discuss receiver operator characteristic curves. ROC curves build on the idea of a confusion matrix but provide us with deeper information that lets us improve our models to a greater degree.
 
-## Scenario:
+## Scenario
 
-Throughout this module, we’ll be using the following example scenario to explain and practice working with ROC curves.
+Throughout this module, we'll be using the following example scenario to explain and practice working with ROC curves.
 
 Your avalanche-rescue charity has successfully built a machine learning model that can estimate whether an object detected by lightweight sensors is a hiker or a natural object, such as a tree or a rock. This lets you keep track of how many people are on the mountain, so you know whether a rescue team is needed when an avalanche strikes. The model does reasonably well, though you wonder if there's room for improvement. Internally, the model must make a binary decision as to whether an object is a hiker or not, but this is based on probabilities. Can this decision-making process be tweaked to improve its performance?
 

learn-pr/azure/optimize-model-performance-roc-auc/includes/2-receiver-operator-characteristic-curve.md

@@ -2,7 +2,7 @@ Classification models must assign a sample to a category. For example, it must u
 
 We can improve classification models in many ways. For example, we can ensure our data are balanced, clean, and scaled. We can also alter our model architecture and use hyperparameters to squeeze as much performance as we possibly can out of our data and architecture. Eventually, we find no better way to improve performance on our test (or hold-out) set and declare our model ready.
 
-Model tuning to this point can be complex, but we can use a final simple step to further improve how well our model works. To understand this, though, we need to go back to basics.
+Model tuning to this point can be complex, but we can use a final step to further improve how well our model works. To understand this, though, we need to go back to basics.
 
 ## Probabilities and categories
 
@@ -23,17 +23,17 @@ We can calculate some handy characteristics from the confusion matrix. Two popul
 
 Looking at true positive and false positive rates can help us understand a model's performance.
 
-Consider our hiker example. Ideally, the true positive rate is very high, and the false positive rate is very low, because this means that the model identifies hikers well and doesn't identify trees as hikers very often. Yet, if the true positive rate is very high, but the false positive rate is also very high, then the model is biased; it's identifying almost everything it encounters as hiker. Similarly, we don't want a model with a low true positive rate, because then when the model encounters a hiker, it'll label them as a tree.
+Consider our hiker example. Ideally, the true positive rate is very high, and the false positive rate is very low. This means that the model identifies hikers well and doesn't identify trees as hikers very often. Yet, if the true positive rate is very high, but the false positive rate is also very high, then the model is biased; it's identifying almost everything it encounters as hiker. Similarly, we don't want a model with a low true positive rate, because then when the model encounters a hiker, it'll label them as a tree.
 
 ## ROC curves
 
-Receiver operator characteristic (ROC) curves are a graph where we plot true positive rate versus false positive rate.
+Receiver operator characteristic curves are a graph where we plot true positive rate versus false positive rate.
 
 ROC curves can be confusing for beginners for two main reasons. The first reason is that beginners know that a model only has one value for true positive and true negative rates, so an ROC plot must look like this:
 
 ![Receiver operator characteristic curve graph with one plot point.](../media/roc-graph.png)
 
-If you're also thinking this, you're right. A trained model only produces one point. However, remember that our models have a threshold—normally 50%—that's used to decide whether the true (hiker) or false (tree) label should be used. If we change this threshold to 30% and recalculate true positive and false positive rates, we get another point:
+If you're also thinking this, you're right. A trained model only produces one point. However, remember that our models have a threshold-normally 50%-that's used to decide whether the true (hiker) or false (tree) label should be used. If we change this threshold to 30% and recalculate true positive and false positive rates, we get another point:
 
 ![Receiver operator characteristic curve graph with two plot points.](../media/roc-graph-2.png)