Merge pull request #18 from MicrosoftCloudEssentials-LearningHub/bestpractices7of9

best practices overview 6 of 7 workloads

at the end

Author: brown9804

README.md

@@ -173,7 +173,7 @@ Click to read more about [Microsoft Purview for Fabric - Overview](./Workloads-S
 - **Microsoft [Fabric Capacity Metrics](https://github.com/MicrosoftCloudEssentials-LearningHub/Fabric-EnterpriseFramework/blob/main/Monitoring-Observability/MonitorUsage.md#microsoft-fabric-capacity-metrics-app) app**: Powerful tool for administrators to `monitor and manage their capacity usage`. It provides detailed insights into `capacity utilization, throttling, and system events, helping to optimize performance and resource allocation`. By tracking these metrics, admins can make informed decisions to ensure efficient use of resources.
 - **Admin Monitoring**: Configure and use the [Admin Monitoring Workspace](https://github.com/MicrosoftCloudEssentials-LearningHub/Fabric-EnterpriseFramework/blob/main/Monitoring-Observability/MonitorUsage.md#admin-monitoring) it's a centralized hub for `tracking and analyzing usage metrics across the organization`. It includes `pre-built reports and semantic models that provide insights into feature adoption, performance, and compliance`. This workspace helps administrators maintain the health and efficiency of their Fabric environment by offering a comprehensive `view of usage patterns and system events`.
 - **Monitor Hub**: Access and utilize the [Monitor Hub](https://github.com/MicrosoftCloudEssentials-LearningHub/Fabric-EnterpriseFramework/blob/main/Monitoring-Observability/MonitorUsage.md#monitor-hub). Allows users to `view and track the status of activities across all workspaces they have permissions for`. It provides a detailed overview of operations, `including dataset refreshes, Spark job runs, and other activities`. With features like historical views, customizable displays, and filtering options, the Monitor Hub helps ensure smooth operations and timely interventions when needed.
-- **Event Hub Integration**: Use Event Hub to capture and analyze events for real-time monitoring. For example, leverage it for [Automating pipeline execution with Activator](./Monitoring-Observability/FabricActivatorRulePipeline/)
+- **Event Hub Integration**: Use Event Hub to capture and analyze events for real-time monitoring. For example, leverage it for [Automating pipeline execution with Activator](./Workloads-Specific/RealTimeIntelligence/FabricActivatorRulePipeline/)
 - **Alerting**: Configure alerts for critical events and thresholds to ensure timely responses to issues. For example, [Steps to Configure Capacity Alerts](./Monitoring-Observability/StepsCapacityAlert.md)
 
 ## Cost Management 
@@ -202,12 +202,10 @@ Click to read more about [Microsoft Purview for Fabric - Overview](./Workloads-S
 - [Azure Data Factory (ADF) - Best Practices Overview](./Workloads-Specific/DataFactory/BestPractices.md)
 - [Data Engineering - Best Practices Overview](./Workloads-Specific/DataEngineering/BestPractices.md)
 - [Data Warehouse - Best Practices Overview](./Workloads-Specific/DataWarehouse/BestPractices.md) 
-- [Data Science - Best Practices Overview](./Workloads-Specific/DataScience/BestPractices.md) - in progress
-- [Real-Time Intelligence - Best Practices Overview](./Workloads-Specific/RealTimeIntelligence/BestPractices.md) - in progress
+- [Data Science - Best Practices Overview](./Workloads-Specific/DataScience/BestPractices.md) 
+- [Real-Time Intelligence - Best Practices Overview](./Workloads-Specific/RealTimeIntelligence/BestPractices.md)
 - [Power Bi - Best Practices Overview](./Workloads-Specific/PowerBi/BestPractices.md)
-- [Copilot - Best Practices Overview](./Workloads-Specific/Copilot/BestPractices.md) - in progress
 - [Purview - Best Practices Overview](./Workloads-Specific/Purview/BestPractices.md) - in progress
-- [OneLake - Best Practices Overview](./Workloads-Specific/OneLake/BestPractices.md) - in progress
 
 <div align="center">
   <h3 style="color: #4CAF50;">Total Visitors</h3>

Workloads-Specific/DataScience/BestPractices.md

@@ -13,8 +13,47 @@ Last updated: 2025-05-03
 <details>
 <summary><b>List of References</b> (Click to expand)</summary>
 
+- [What is Data Science in Microsoft Fabric?](https://learn.microsoft.com/en-us/fabric/data-science/data-science-overview)
+- [Data Science documentation in Microsoft Fabric](https://learn.microsoft.com/en-us/fabric/data-science/)
+
+</details>
+
+<details>
+<summary><b>Table of Content</b> (Click to expand)</summary>
+
+- [ML Model Management](#ml-model-management)
+- [Experiment Tracking & Management](#experiment-tracking--management)
+- [Reproducible Environments](#reproducible-environments)
+- [Data Agent Preview Usage](#data-agent-preview-usage)
+
 </details>
 
+> Ensure that your data science workflows in Microsoft Fabric are built for rapid experimentation, efficient model management, and seamless deployment. Each element should be managed with clear versioning, detailed documentation, and reproducible environments, enabling a smooth transition from experimentation to production.
+
+<div align="center">
+  <img src="https://github.com/user-attachments/assets/f86cdba7-e9a6-4ce1-8dcc-912b7f438398" alt="Centered Image" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
+</div>
+
+## ML Model Management 
+
+> Use model registries integrated within Fabric to store and version your models. Include a descriptive README, link relevant experiment IDs, and attach performance metrics such as accuracy, AUC, and confusion matrices. For example, link your production-ready model (v#.#) from a registered repository along with its associated validation metrics and deployment instructions.  
+
+## Experiment Tracking & Management 
+
+> Set up an experiment dashboard that automatically logs training runs. For instance, record runs with various hyperparameter combinations, tag them with unique identifiers, and visualize comparative metrics over multiple iterations. This dashboard can help you decide whether a model trained with early stopping or one with higher epochs best meets performance goals.
+                                                                                                       
+## Reproducible Environments 
+
+> Create an environment file (e.g., Conda `environment.yml`) that lists all required Python packages and their versions. For example, specify TensorFlow 2.9, scikit-learn 1.0, and other dependencies so that every data scientist and deployment pipeline uses the exact setup. Use Microsoft Fabric workspaces to segregate development and production environments, ensuring that models are trained and evaluated in a consistent setting.
+
+<https://github.com/user-attachments/assets/fcce754d-afd3-4267-aa0f-bba87c0a3089>
+
+## Data Agent (Preview) Usage 
+
+> Integrate the Data Agent into your pipeline to automatically validate incoming datasets for completeness and consistency. For instance, set up rules that flag missing data or out-of-range values and trigger notifications when anomalies are detected. Track and document these incidents to help refine the agent’s calibration, ensuring that data passing to your experiments meets quality standards.  
+
+Click to read [Demonstration: Data Agents in Microsoft Fabric](./Data_Agents.md).
+
 <div align="center">
   <h3 style="color: #4CAF50;">Total Visitors</h3>
   <img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>

Workloads-Specific/Copilot/BestPractices.md renamed to Workloads-Specific/DataScience/How_AutoML/README.md

@@ -1,4 +1,4 @@
-# Copilot - Best Practices Overview
+# Demonstration: How to train a ML model with AutoML
 
 Costa Rica
 
@@ -10,10 +10,15 @@ Last updated: 2025-05-03
 
 ----------
 
-<details>
-<summary><b>List of References</b> (Click to expand)</summary>
+> How to create an experiment to train a ML model with AutoML:
 
-</details>
+<https://github.com/user-attachments/assets/4c73eaaa-cf03-47cf-807b-69007c8df704>
+
+Click to see notebook generated [Train a ML model with AutoML](./Train_MLmodel_AutoML.ipynb)
+
+> Run the notebook the generated: 
+
+<https://github.com/user-attachments/assets/6dfedbac-beb7-4025-9a42-f98dade7f431>
 
 <div align="center">
   <h3 style="color: #4CAF50;">Total Visitors</h3>