Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
92059e4
best practices overview 7 of 9 workloads
brown9804 May 3, 2025
94a5659
Merge 92059e4f8658fd9d35db2c24b12ab998bb2cf8cb into 73be35c1bcbc87389…
brown9804 May 3, 2025
2eb6052
Fix Markdown syntax issues
github-actions[bot] May 3, 2025
030fcc3
title
brown9804 May 3, 2025
1232088
title
brown9804 May 3, 2025
6acba00
+ overview visual guidance
brown9804 May 3, 2025
bf883b8
Merge 6acba00c1907caef22f3fe94ae6457f81f8653e0 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
5067c15
Fix Markdown syntax issues
github-actions[bot] May 3, 2025
9127b78
title
brown9804 May 3, 2025
c3fc909
Merge 9127b78b247f58d98acad4fafefb384eb1556187 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
007f64c
ds in place
brown9804 May 3, 2025
1c3673c
Merge 007f64c24e72f1c0747cb6befc2caddd2cd655d3 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
2c62602
path
brown9804 May 3, 2025
e411f90
Merge 2c62602c09b1117f1741b12e5d884b94f25df06b into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
3738161
moved
brown9804 May 3, 2025
0707ea1
Merge 3738161adc367aac45a24ab4f78a6fad4d835f40 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
193e679
missing how looks when run
brown9804 May 3, 2025
0c287ca
Merge 193e6790401586f719b8b2ff82a63653b24cae0b into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
89b7299
Fix Markdown syntax issues
github-actions[bot] May 3, 2025
1d201b8
sample added
brown9804 May 3, 2025
c24d200
Merge 1d201b8111e87ea2a3341b9f6069dd0b523ce65b into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
332b653
Fix notebook format issues
github-actions[bot] May 3, 2025
6263f41
quick demo
brown9804 May 3, 2025
799bfca
Merge 6263f41f37e832e1bcd9d325777ad19775863206 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
1dd2fee
Fix Markdown syntax issues
github-actions[bot] May 3, 2025
2cd2086
ds workload
brown9804 May 3, 2025
984b13c
Merge 2cd208677c8f26bd979dc3271d0d8865ccf876b1 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
a95cc80
in progress
brown9804 May 3, 2025
fea4ea0
Merge a95cc80efa4b5201b8d365eeaa0ee6c929c36142 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
cf17ab6
Fix Markdown syntax issues
github-actions[bot] May 3, 2025
a3be59f
no needed those 2
brown9804 May 3, 2025
4e6ca92
Merge a3be59fdf3220f7ce668527d2e3a0e64ea9d6729 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
fb07f4d
no need
brown9804 May 3, 2025
c89596b
Merge fb07f4d4408a7795c663d4c9bb5766a972c805e6 into 1232088c1ae8c7fe6…
brown9804 May 3, 2025
daf0dd1
no need
brown9804 May 3, 2025
65701ac
moved
brown9804 May 3, 2025
75ddc5b
moved
brown9804 May 3, 2025
17e2e02
moved
brown9804 May 3, 2025
59ae01f
moved
brown9804 May 3, 2025
dbe7dc6
ref path changed
brown9804 May 3, 2025
dcdfa2b
title c
brown9804 May 3, 2025
8a82f55
Fix Markdown syntax issues
github-actions[bot] May 3, 2025
fb84aa4
in place
brown9804 May 3, 2025
3b56667
pending purvire
brown9804 May 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ Click to read more about [Microsoft Purview for Fabric - Overview](./Workloads-S
- **Microsoft [Fabric Capacity Metrics](https://github.com/MicrosoftCloudEssentials-LearningHub/Fabric-EnterpriseFramework/blob/main/Monitoring-Observability/MonitorUsage.md#microsoft-fabric-capacity-metrics-app) app**: Powerful tool for administrators to `monitor and manage their capacity usage`. It provides detailed insights into `capacity utilization, throttling, and system events, helping to optimize performance and resource allocation`. By tracking these metrics, admins can make informed decisions to ensure efficient use of resources.
- **Admin Monitoring**: Configure and use the [Admin Monitoring Workspace](https://github.com/MicrosoftCloudEssentials-LearningHub/Fabric-EnterpriseFramework/blob/main/Monitoring-Observability/MonitorUsage.md#admin-monitoring) it's a centralized hub for `tracking and analyzing usage metrics across the organization`. It includes `pre-built reports and semantic models that provide insights into feature adoption, performance, and compliance`. This workspace helps administrators maintain the health and efficiency of their Fabric environment by offering a comprehensive `view of usage patterns and system events`.
- **Monitor Hub**: Access and utilize the [Monitor Hub](https://github.com/MicrosoftCloudEssentials-LearningHub/Fabric-EnterpriseFramework/blob/main/Monitoring-Observability/MonitorUsage.md#monitor-hub). Allows users to `view and track the status of activities across all workspaces they have permissions for`. It provides a detailed overview of operations, `including dataset refreshes, Spark job runs, and other activities`. With features like historical views, customizable displays, and filtering options, the Monitor Hub helps ensure smooth operations and timely interventions when needed.
- **Event Hub Integration**: Use Event Hub to capture and analyze events for real-time monitoring. For example, leverage it for [Automating pipeline execution with Activator](./Monitoring-Observability/FabricActivatorRulePipeline/)
- **Event Hub Integration**: Use Event Hub to capture and analyze events for real-time monitoring. For example, leverage it for [Automating pipeline execution with Activator](./Workloads-Specific/RealTimeIntelligence/FabricActivatorRulePipeline/)
- **Alerting**: Configure alerts for critical events and thresholds to ensure timely responses to issues. For example, [Steps to Configure Capacity Alerts](./Monitoring-Observability/StepsCapacityAlert.md)

## Cost Management
Expand Down Expand Up @@ -202,12 +202,10 @@ Click to read more about [Microsoft Purview for Fabric - Overview](./Workloads-S
- [Azure Data Factory (ADF) - Best Practices Overview](./Workloads-Specific/DataFactory/BestPractices.md)
- [Data Engineering - Best Practices Overview](./Workloads-Specific/DataEngineering/BestPractices.md)
- [Data Warehouse - Best Practices Overview](./Workloads-Specific/DataWarehouse/BestPractices.md)
- [Data Science - Best Practices Overview](./Workloads-Specific/DataScience/BestPractices.md) - in progress
- [Real-Time Intelligence - Best Practices Overview](./Workloads-Specific/RealTimeIntelligence/BestPractices.md) - in progress
- [Data Science - Best Practices Overview](./Workloads-Specific/DataScience/BestPractices.md)
- [Real-Time Intelligence - Best Practices Overview](./Workloads-Specific/RealTimeIntelligence/BestPractices.md)
- [Power Bi - Best Practices Overview](./Workloads-Specific/PowerBi/BestPractices.md)
- [Copilot - Best Practices Overview](./Workloads-Specific/Copilot/BestPractices.md) - in progress
- [Purview - Best Practices Overview](./Workloads-Specific/Purview/BestPractices.md) - in progress
- [OneLake - Best Practices Overview](./Workloads-Specific/OneLake/BestPractices.md) - in progress

<div align="center">
<h3 style="color: #4CAF50;">Total Visitors</h3>
Expand Down
2 changes: 1 addition & 1 deletion Workloads-Specific/DataScience/AI_integration/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Demostration: How to integrate AI in Microsoft Fabric
# Demonstration: How to integrate AI in Microsoft Fabric

Costa Rica

Expand Down
39 changes: 39 additions & 0 deletions Workloads-Specific/DataScience/BestPractices.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,47 @@ Last updated: 2025-05-03
<details>
<summary><b>List of References</b> (Click to expand)</summary>

- [What is Data Science in Microsoft Fabric?](https://learn.microsoft.com/en-us/fabric/data-science/data-science-overview)
- [Data Science documentation in Microsoft Fabric](https://learn.microsoft.com/en-us/fabric/data-science/)

</details>

<details>
<summary><b>Table of Content</b> (Click to expand)</summary>

- [ML Model Management](#ml-model-management)
- [Experiment Tracking & Management](#experiment-tracking--management)
- [Reproducible Environments](#reproducible-environments)
- [Data Agent Preview Usage](#data-agent-preview-usage)

</details>

> Ensure that your data science workflows in Microsoft Fabric are built for rapid experimentation, efficient model management, and seamless deployment. Each element should be managed with clear versioning, detailed documentation, and reproducible environments, enabling a smooth transition from experimentation to production.

<div align="center">
<img src="https://github.com/user-attachments/assets/f86cdba7-e9a6-4ce1-8dcc-912b7f438398" alt="Centered Image" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
</div>

## ML Model Management

> Use model registries integrated within Fabric to store and version your models. Include a descriptive README, link relevant experiment IDs, and attach performance metrics such as accuracy, AUC, and confusion matrices. For example, link your production-ready model (v#.#) from a registered repository along with its associated validation metrics and deployment instructions.

## Experiment Tracking & Management

> Set up an experiment dashboard that automatically logs training runs. For instance, record runs with various hyperparameter combinations, tag them with unique identifiers, and visualize comparative metrics over multiple iterations. This dashboard can help you decide whether a model trained with early stopping or one with higher epochs best meets performance goals.

## Reproducible Environments

> Create an environment file (e.g., Conda `environment.yml`) that lists all required Python packages and their versions. For example, specify TensorFlow 2.9, scikit-learn 1.0, and other dependencies so that every data scientist and deployment pipeline uses the exact setup. Use Microsoft Fabric workspaces to segregate development and production environments, ensuring that models are trained and evaluated in a consistent setting.

<https://github.com/user-attachments/assets/fcce754d-afd3-4267-aa0f-bba87c0a3089>

## Data Agent (Preview) Usage

> Integrate the Data Agent into your pipeline to automatically validate incoming datasets for completeness and consistency. For instance, set up rules that flag missing data or out-of-range values and trigger notifications when anomalies are detected. Track and document these incidents to help refine the agent’s calibration, ensuring that data passing to your experiments meets quality standards.

Click to read [Demonstration: Data Agents in Microsoft Fabric](./Data_Agents.md).

<div align="center">
<h3 style="color: #4CAF50;">Total Visitors</h3>
<img src="https://profile-counter.glitch.me/brown9804/count.svg" alt="Visitor Count" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
Expand Down
2 changes: 1 addition & 1 deletion Workloads-Specific/DataScience/Data_Agents.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Demostration: Data Agents in Microsoft Fabric (Preview)
# Demonstration: Data Agents in Microsoft Fabric (Preview)

Costa Rica

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copilot - Best Practices Overview
# Demonstration: How to train a ML model with AutoML

Costa Rica

Expand All @@ -10,10 +10,15 @@ Last updated: 2025-05-03

----------

<details>
<summary><b>List of References</b> (Click to expand)</summary>
> How to create an experiment to train a ML model with AutoML:

</details>
<https://github.com/user-attachments/assets/4c73eaaa-cf03-47cf-807b-69007c8df704>

Click to see notebook generated [Train a ML model with AutoML](./Train_MLmodel_AutoML.ipynb)

> Run the notebook the generated:

<https://github.com/user-attachments/assets/6dfedbac-beb7-4025-9a42-f98dade7f431>

<div align="center">
<h3 style="color: #4CAF50;">Total Visitors</h3>
Expand Down
Loading
Loading