Skip to content

Commit 997d928

Browse files
brown9804brown9804
authored
overview in place
1 parent badd7a4 commit 997d928

File tree

1 file changed

+175
-6
lines changed

1 file changed

+175
-6
lines changed

‎README.md‎

Lines changed: 175 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,11 @@ Last updated: 2025-04-29
2323
<details>
2424
<summary><b>Table of Content </b> (Click to expand)</summary>
2525

26+
- [Workspace](#workspace)
27+
- [Authoring](#authoring)
28+
- [Assets](#assets)
29+
- [Manage](#manage)
30+
2631
</details>
2732

2833
> Azure Machine Learning (PaaS) is a cloud-based platform from Microsoft designed to help `data scientists and machine learning engineers build, train, deploy, and manage machine learning models at scale`. It supports the `entire machine learning lifecycle, from data preparation and experimentation to deployment and monitoring.` It provides powerful tools for `both code-first and low-code users`, including Jupyter notebooks, drag-and-drop interfaces, and automated machine learning (AutoML). `Azure ML integrates seamlessly with other Azure services and supports popular frameworks like TensorFlow, PyTorch, and Scikit-learn.`
@@ -58,11 +63,6 @@ https://github.com/user-attachments/assets/c199156f-96cf-4ed0-a8b5-c88db3e7a552
5863
<img src="https://github.com/user-attachments/assets/f3a987a8-ec59-42de-9e56-29c41c8e55a9" alt="Centered Image" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
5964
</div>
6065

61-
Read more about [Endpoints for inference in production](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints?view=azureml-api-2)
62-
63-
<div align="center">
64-
<img src="https://github.com/user-attachments/assets/aa5a0671-e9b8-4ae1-bd69-5098218b63d5" alt="Centered Image" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
65-
</div>
6666

6767
## Authoring
6868

@@ -94,11 +94,124 @@ Read more about [Endpoints for inference in production](https://learn.microsoft.
9494
9595
</details>
9696

97-
9897
## Assets
9998

99+
> The following diagram illustrates how you can use a single Environment object in both your run configuration (for training) and your inference and deployment configuration (for web service deployments).
100+
101+
<div align="center">
102+
<img src="https://github.com/user-attachments/assets/2bc9ccd7-ebe9-478e-ad7d-e10eee0d1892" alt="Centered Image" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
103+
</div>
104+
105+
<details>
106+
<summary><strong>Data</strong></summary>
107+
108+
> **Datasets** are foundational assets in machine learning workflows. In Azure Machine Learning, registered datasets are used across experiments, pipelines, and jobs to ensure consistency and reproducibility. These datasets can be versioned and stored in Azure Blob Storage or other supported data sources. Azure ML supports various data formats, including tabular data (e.g., CSV, Parquet), file-based data (e.g., images, text files), and URI-based references. By registering datasets, teams can track data lineage, manage access, and ensure that models are trained and evaluated on consistent data versions.
109+
110+
1. Prepare your data in a supported format (e.g., CSV, JSON, images).
111+
2. Upload the data to Azure Blob Storage or another supported location.
112+
3. Register the dataset using the Azure ML SDK, CLI, or Studio.
113+
4. Version the dataset to track changes over time.
114+
5. Use the dataset in jobs, pipelines, or experiments to ensure reproducibility.
115+
116+
</details>
117+
118+
<details>
119+
<summary><strong>Jobs</strong></summary>
120+
121+
> **Jobs** represent individual executions of scripts, components, or pipelines in Azure ML. Each job captures inputs, outputs, logs, metrics, and environment details. Jobs can be triggered manually, on a schedule, or as part of a pipeline. They provide full traceability and monitoring for ML experiments and production workflows.
122+
123+
1. Define your script or component (e.g., training or evaluation logic).
124+
2. Specify inputs (datasets, parameters) and expected outputs.
125+
3. Configure the environment (Docker image, Conda dependencies).
126+
4. Submit the job using the Azure ML SDK, CLI, or Studio interface.
127+
5. Monitor the job’s progress, view logs, and analyze metrics.
128+
129+
</details>
130+
131+
<details>
132+
<summary><strong>Components</strong></summary>
133+
134+
> **Components** are modular, reusable units of computation in Azure ML. They encapsulate specific tasks such as data preprocessing, model training, or evaluation. Defined using YAML, components support versioning and parameterization, and are the core building blocks of pipelines.
135+
136+
1. Write a script that performs a specific task (e.g., `train.py`).
137+
2. Create a YAML file defining the component (inputs, outputs, environment).
138+
3. Register the component in Azure ML.
139+
4. Use the component in pipelines or jobs.
140+
5. Version the component to enable reuse and traceability.
141+
142+
</details>
143+
144+
<details>
145+
<summary><strong>Pipelines</strong></summary>
146+
147+
> **Pipelines** orchestrate multiple components into a complete machine learning workflow. They support parallelism, conditional logic, and reuse. Pipelines can be triggered via REST API, CLI, or SDK, and are ideal for automating ML workflows from data ingestion to deployment.
148+
149+
1. Define components for each stage (e.g., preprocessing, training, evaluation).
150+
2. Chain components together using the Azure ML SDK.
151+
3. Specify data flow and dependencies between components.
152+
4. Submit the pipeline as a job.
153+
5. Monitor execution and reuse pipelines for automation and CI/CD.
154+
155+
</details>
156+
157+
<details>
158+
<summary><strong>Environments</strong></summary>
159+
160+
> **Environments** define the runtime configuration for jobs and components. They include Docker base images, Conda dependencies, and environment variables. Environments ensure consistency across development, training, and production stages.
161+
162+
1. Create a Conda YAML file or specify a Docker image.
163+
2. Register the environment in Azure ML.
164+
3. Attach the environment to jobs or components.
165+
4. Version environments to maintain reproducibility.
166+
5. Use consistent environments across all stages of the ML lifecycle.
167+
168+
</details>
169+
170+
<details>
171+
<summary><strong>Models</strong></summary>
172+
173+
> **Models** are trained artifacts stored in the Azure ML workspace. They can be versioned, registered, and deployed to endpoints. Each model includes metadata, lineage, and evaluation metrics, supporting traceability and governance.
174+
175+
1. Train a model using a job or pipeline.
176+
2. Register the model in the Azure ML workspace.
177+
3. Attach metadata such as metrics, tags, and lineage.
178+
4. Version the model to track improvements.
179+
5. Deploy the model to an endpoint for inference.
180+
181+
</details>
182+
183+
<details>
184+
<summary><strong>Endpoints</strong></summary>
185+
186+
> **Endpoints** expose deployed models as REST APIs for real-time (online) or batch inference. They support authentication, virtual network (VNet) integration, and traffic splitting for A/B testing or gradual rollouts.
187+
188+
1. Choose a deployment target (real-time or batch).
189+
2. Create an inference configuration (entry script, environment).
190+
3. Deploy the model to an endpoint.
191+
4. Secure the endpoint with authentication and networking rules.
192+
5. Monitor and scale the endpoint as needed.
193+
194+
</details>
195+
196+
Read more about [Endpoints for inference in production](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints?view=azureml-api-2)
197+
198+
<img src="https://github.com/user-attachments/assets/aa5a0671-e9b8-4ae1-bd69-5098218b63d5" alt="Centered Image" style="border: 2px solid #4CAF50; border-radius: 5px; padding: 5px;"/>
199+
200+
100201
## Manage
101202

203+
204+
<details>
205+
<summary><strong>Compute</strong></summary>
206+
207+
> **Compute** resources are the backbone of Azure Machine Learning workloads. This section allows you to create, manage, and monitor compute instances and clusters used for development, training, and inference. It supports various compute types including personal VMs, scalable clusters, and attached external resources.
208+
209+
1. Navigate to **Manage > Compute** in Azure ML Studio.
210+
2. Choose the type of compute: Instance (for development), Cluster (for training), or Inference Cluster (for deployment).
211+
3. Configure the compute settings (VM size, scaling, idle shutdown).
212+
4. Attach external compute if needed (e.g., Azure Databricks, AKS).
213+
5. Monitor usage, status, and logs from the same interface.
214+
102215
| Compute Type | Description | Purpose | Ideal Use Cases | Key Features |
103216
|----------------------|-------------|---------|------------------|---------------|
104217
| **Compute Instances** | Preconfigured VMs (CPU/GPU) with tools like VS Code, JupyterLab, and RStudio. | Interactive development and experimentation. | Data exploration, model prototyping, debugging. | Auto-shutdown, integrated notebooks, preinstalled ML frameworks. |
@@ -108,6 +221,62 @@ Read more about [Endpoints for inference in production](https://learn.microsoft.
108221
| **Serverless Instances** | Lightweight, on-demand compute (e.g., Azure Container Instances). | Quick testing and low-scale inference. | Temporary model deployment, dev/test environments. | No infrastructure management, fast startup, cost-effective. |
109222

110223

224+
</details>
225+
226+
<details>
227+
<summary><strong>Monitoring</strong></summary>
228+
229+
> **Monitoring** provides visibility into the performance and health of your ML assets. It includes logs, metrics, and telemetry for jobs, pipelines, and deployed endpoints. This helps ensure reliability, detect anomalies, and maintain model quality over time.
230+
231+
1. Go to **Manage > Monitoring**.
232+
2. Select the asset you want to monitor (e.g., job, endpoint).
233+
3. View logs, metrics, and telemetry data.
234+
4. Set up alerts for failures or performance thresholds.
235+
5. Use insights to debug issues or optimize performance.
236+
237+
</details>
238+
239+
<details>
240+
<summary><strong>Data Labeling</strong></summary>
241+
242+
> **Data Labeling** enables you to create and manage labeling projects for supervised learning. It supports image, text, and tabular data, and allows collaboration with human labelers or integration with labeling services.
243+
244+
1. Open **Manage > Data Labeling**.
245+
2. Create a new labeling project and select the data type.
246+
3. Upload or link the dataset to be labeled.
247+
4. Assign labeling tasks to users or services.
248+
5. Export the labeled dataset for training or evaluation.
249+
250+
</details>
251+
252+
<details>
253+
<summary><strong>Linked Services (Preview)</strong></summary>
254+
255+
> **Linked Services** allow you to connect your Azure ML workspace to external data sources like Azure Data Lake, Blob Storage, or SQL databases. This simplifies data access and centralizes configuration for use in datasets and pipelines.
256+
257+
1. Navigate to **Manage > Linked Services**.
258+
2. Add a new linked service by selecting the data source type.
259+
3. Provide connection details and authentication credentials.
260+
4. Test the connection to ensure access.
261+
5. Use the linked service in datasets or pipeline steps.
262+
263+
</details>
264+
265+
<details>
266+
<summary><strong>Connections (Preview)</strong></summary>
267+
268+
> **Connections** manage secure access to external systems and APIs. They store credentials and authentication methods, enabling secure and reusable access in jobs and pipelines.
269+
270+
1. Go to **Manage > Connections**.
271+
2. Create a new connection (e.g., to a REST API or database).
272+
3. Define the authentication method (e.g., service principal, managed identity).
273+
4. Save and test the connection.
274+
5. Reference the connection in your components or pipelines.
275+
276+
</details>
277+
278+
279+
111280

112281

113282
<div align="center">

0 commit comments

Comments
 (0)