Merge pull request #50297 from MicrosoftDocs/NEW-genaiops

New genaiops module

Author: JillGrant615

learn-pr/wwl-data-ai/monitor-generative-ai-app/1-introduction.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.introduction
+title: Introduction
+metadata:
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Introduction
+  description: It's important to monitor your generative AI application.
+  author: wwlpublish
+  ms.author: madiepev
+  ms.date: 04/30/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 1
+content: |
+  [!include[](includes/1-introduction.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/2-why-monitor.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.why-monitor
+title: Why do you need to monitor?
+metadata:
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Why do you need to monitor?
+  description: Understand the need for monitoring your generative AI application.
+  author: wwlpublish
+  ms.author: madiepev
+  ms.date: 04/30/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 5
+content: |
+  [!include[](includes/2-why-monitor.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/3-what-to-monitor.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.what-to-monitor
+title: Understand key metrics to monitor
+metadata:
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Understand key metrics to monitor
+  description: Understand which key metrics to monitor for generative AI applications.
+  author: wwlpublish
+  ms.author: madiepev
+  ms.date: 04/30/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 5
+content: |
+  [!include[](includes/3-what-to-monitor.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/4-how-to-monitor.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.how-to-monitor
+title: Explore how to monitor with Azure
+metadata:
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Explore how to monitor with Azure
+  description: Explore how to continuously monitor generative AI applications with Azure.
+  author: wwlpublish
+  ms.author: madiepev
+  ms.date: 04/30/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 5
+content: |
+  [!include[](includes/4-how-to-monitor.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/5-prepare-scripts.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.prepare-scripts
+title: Integrate monitoring into your app
+metadata: 
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Integrate monitoring into your app
+  description: Learn how to instrument your generative AI application with Azure tools to capture trace data, evaluate model performance, and monitor system behavior in real time.
+  author: wwlpublish
+  ms.author: madiepev
+  ms.date: 04/30/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 5
+content: |
+  [!include[](includes/5-prepare-scripts.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/6-informed-decisions.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.informed-decisions
+title: Interpret monitoring results
+metadata: 
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Interpret monitoring results
+  description: Learn how to interpret monitoring results to make informed decisions about your configuration.
+  author: wwlpublish
+  ms.author: madiepev
+  ms.date: 04/30/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 5
+content: |
+  [!include[](includes/6-informed-decisions.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/7-exercise.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.exercise
+title: Exercise - Enable monitoring for a generative AI application
+metadata:
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Exercise - Enable monitoring for a generative AI application 
+  description: Enable monitoring for a generative AI application in Azure.
+  author: wwlpublish
+  ms.author: gmalc
+  ms.date: 01/31/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 30
+content: |
+  [!include[](includes/7-exercise.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/8-knowledge-check.yml

@@ -0,0 +1,52 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.knowledge-check
+title: Knowledge check
+metadata:
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Knowledge Check
+  description: Test your knowledge.
+  author: wwlpublish
+  ms.author: madiepev
+  ms.date: 04/30/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 3
+content: |
+quiz:
+  questions:
+  - content: "What serves as the foundation of continuous monitoring for generative AI applications?"
+    choices:
+    - content: "Online Evaluation"
+      isCorrect: false
+      explanation: "Incorrect. Online Evaluations is an extension of tracing."
+    - content: "Tracing"
+      isCorrect: true
+      explanation: "Correct. Tracing captures the core telemetry data that enables all other monitoring and evaluation."
+    - content: "Application Insights "
+      isCorrect: false
+      explanation: "Incorrect. Application Insights displays trace data but does not generate it."
+  - content: "What does Azure AI Online Evaluation help assess in real time?"
+    choices:
+    - content: "The latency of AI responses."
+      isCorrect: false
+      explanation: "Incorrect. Latency is measured through tracing, not online evaluation."
+    - content: "The quality of AI-generated outputs."
+      isCorrect: true
+      explanation: "Correct. Online evaluation assesses response quality, safety, and relevance in real-time."
+    - content: "The token cost of model deployments."
+      isCorrect: false
+      explanation: "Incorrect. Token usage is tracked via tracing, not part of Online Evaluations."
+  - content: "Which component provides advanced observability tools including custom dashboards and configurable alerting mechanisms?"
+    choices:
+    - content: "Azure Monitor"
+      isCorrect: true
+      explanation: "Correct. Azure Monitor provides dashboards, alerts, and analytics for end-to-end observability."
+    - content: "Azure AI Inference SDK"
+      isCorrect: false
+      explanation: "Incorrect. The Inference SDK is used to run model operations, not for monitoring or observability."
+    - content: "Azure AI Foundry SDK"
+      isCorrect: false
+      explanation: "Incorrect. The Azure AI Foundry SDK manages projects and deployments but doesn’t offer observability tools."
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/9-summary.yml

@@ -0,0 +1,18 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.monitor-generative-ai-app.summary
+title: Summary
+metadata:
+  adobe-target: true
+  prefetch-feature-rollout: true
+  title: Summary
+  description: Summary of planning considerations for AI development on Azure.
+  author: wwlpublish
+  ms.author: gmalc
+  ms.date: 01/31/2025
+  ms.topic: unit
+  ms.collection:
+    - wwl-ai-copilot
+durationInMinutes: 1
+content: |
+  [!include[](includes/9-summary.md)]
+

learn-pr/wwl-data-ai/monitor-generative-ai-app/includes/1-introduction.md

@@ -0,0 +1,17 @@
+In the early stages of working with generative AI, it's common to focus on getting something that works. Whether it's a demo, a prototype, or a proof-of-concept, these milestones can all feel significant. However, making something production-ready is a different challenge.
+
+Without proper monitoring, even seemingly stable generative AI applications can face issues in real-world conditions:
+
+- **Latency** can become unpredictable.
+- **Costs** can increase due to inefficient prompt design or scaling.
+- **Compute resources** aren't aligned with actual usage needs.
+
+Many teams fall into the trap of deploying without fully understanding how their system performs under real conditions. Monitoring transforms guesswork into engineering.
+
+## Understand the use case
+
+Imagine you work for Lakeshore Retail, which sells outdoor gear. The customer support team fields hundreds of inquiries daily about your extensive product lineup, ranging from camping gear to specialized hiking equipment. To enhance response speed and accuracy, they deployed an AI assistant named Trail Guide.
+
+However, deploying a generative AI solution is just the beginning. As an AI engineer, you're asked to implement ongoing monitoring to maintain quality, mitigate risk and safety, and ensure customer satisfaction.
+
+In this module, you learn best practices for monitoring generative AI applications with Azure AI and Azure Monitor. By the end, you're able to proactively monitor AI agents and assistants like Trail Guide and optimize their real-world effectiveness.