You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Azure Machine Learning allows you to integrate with [GitHub Actions](https://docs.github.com/actions) to automate the machine learning lifecycle. Some of the operations you can automate are:
21
21
22
-
* Deployment of Azure Machine Learning infrastructure
23
-
* Data preparation (extract, transform, load operations)
24
-
* Training machine learning models with on-demand scale-out and scale-up
25
-
* Deployment of machine learning models as public or private web services
26
-
* Monitoring deployed machine learning models (such as for performance analysis)
22
+
* Deployment of Azure Machine Learning infrastructure.
23
+
* Data preparation (extract, transform, load operations).
24
+
* Training machine learning models with on-demand scale-out and scale-up.
25
+
* Deployment of machine learning models as public or private web services.
26
+
* Monitoring deployed machine learning models (such as for performance analysis).
27
27
28
-
In this article, you learn about using Azure Machine Learning to set up an end-to-end MLOps pipeline that runs a linear regression to predict taxi fares in NYC. The pipeline is made up of components, each serving different functions, which can be registered with the workspace, versioned, and reused with various inputs and outputs. you're going to be using the [recommended Azure architecture for MLOps](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) and [Azure MLOps (v2) solution accelerator](https://github.com/Azure/mlops-v2) to quickly setup an MLOps project in Azure Machine Learning.
28
+
In this article, you learn about using Azure Machine Learning to set up an end-to-end MLOps pipeline that runs a linear regression to predict taxi fares in NYC. The pipeline is made up of components, each serving different functions, which can be registered with the workspace, versioned, and reused with various inputs and outputs. You'll be using the [recommended Azure architecture for MLOps](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) and [Azure MLOps (v2) solution accelerator](https://github.com/Azure/mlops-v2) to quickly setup an MLOps project in Azure Machine Learning.
29
29
30
30
> [!TIP]
31
31
> We recommend you understand some of the [recommended Azure architectures](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) for MLOps before implementing any solution. You'll need to pick the best architecture for your given Machine learning project.
@@ -144,10 +144,10 @@ Before you can set up an MLOps project with Machine Learning, you need to set up
@@ -161,7 +161,7 @@ This step deploys the training pipeline to the Machine Learning workspace create
161
161
> [!TIP]
162
162
> Make sure you understand the [Architectural Patterns](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) of the solution accelerator before you checkout the MLOps v2 repo and deploy the infrastructure. In examples you'll use the [classical ML project type](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2#classical-machine-learning-architecture).
1. Go to your repository and select the `config-infra-prod.yml` file in the root. Change the following parameters to your liking, and then **commit** the changes.
166
166
167
167
This config file uses the namespace and postfix values the names of the artifacts to ensure uniqueness. Update the following section in the config to your liking. Default values and settings in the files are show below:
@@ -177,60 +177,60 @@ This step deploys the training pipeline to the Machine Learning workspace create
177
177
enable_monitoring: false
178
178
```
179
179
> [!NOTE]
180
-
> If you are running a Deep Learning workload such as CV or NLP, ensure your GPU compute is available in your deployment zone.
181
-
> The enable_monitoring flag in these files defaults to False. Enabling this flag will add additional elements to the deployment to support Azure ML monitoring based on https://github.com/microsoft/AzureML-Observability. This will include an ADX cluster and increase the deployment time and cost of the MLOps solution.
180
+
> If you are running a deep learning workload such as CV or NLP, ensure your GPU compute is available in your deployment zone.
181
+
> The enable_monitoring flag in these files defaults to False. Enabling this flag will add additional elements to the deployment to support Azure Machine Learning monitoring based on https://github.com/microsoft/AzureML-Observability. This will include an ADX cluster and increase the deployment time and cost of the MLOps solution.
182
182
183
-
### Deploy Machine Learning infrastructure
183
+
### Deploy machine learning infrastructure
184
184
185
185
1. In your GitHub project repository (ex: taxi-fare-regression), select **Actions**
This displays the pre-defined GitHub workflows associated with your project. For a classical machine learning project, the available workflows look similar to this:
189
+
This displays the pre-defined GitHub workflows associated with your project. For a classical machine learning project, the available workflows look similar to this:
1. On the right side of the page, select **Run workflow** and select the branch to run the workflow on. This may deploy Dev Infrastructure if you've created a dev branch or Prod infrastructure if deploying from main. Monitor the workflow for successful completion.
198
198
199
199

200
200
201
-
1. When the pipeline has complete successfully, you can find your Azure ML Workspace and associated resources by logging in to the Azure Portal. Next, a model training and scoring pipelines will be deployed into the new Machine Learning environment.
201
+
1. When the pipeline has complete successfully, you can find your Azure ML Workspace and associated resources by logging in to the Azure portal. Next, a model training and scoring pipelines will be deployed into the new machine learning environment.
202
202
203
203
204
-
## Sample Training and Deployment Scenario
204
+
## Sample training and deployment scenario
205
205
206
206
The solution accelerator includes code and data fora sample end-to-end machine learning pipeline which runs a linear regression to predict taxi faresin NYC. The pipeline is made up of components, each serving different functions, which can be registered with the workspace, versioned, and reused with various inputs and outputs. Sample pipelines and workflows for the Computer Vision and NLP scenarios will have different steps and deployment steps.
207
207
208
208
This training pipeline contains the following steps:
209
209
210
210
**Prepare Data**
211
211
- This component takes multiple taxi datasets (yellow and green) and merges/filters the data, and prepare the train/val and evaluation datasets.
212
-
- Input: Local data under `./data/` (multiple .csv files)
213
-
- Output: Single prepared dataset (.csv) and train/val/test datasets.
212
+
- **Input:** Local data under `./data/` (multiple .csv files).
213
+
- **Output:** Single prepared dataset (.csv) and train/val/test datasets.
214
214
215
215
**Train Model**
216
216
- This component trains a Linear Regressor with the training set.
217
-
- Input: Training dataset
218
-
- Output: Trained model (pickle format)
217
+
- **Input:** Training dataset.
218
+
- **Output:** Trained model (pickle format).
219
219
220
220
**Evaluate Model**
221
221
- This component uses the trained model to predict taxi fares on the test set.
222
-
- Input: ML model and Test dataset
223
-
- Output: Performance of model and a deploy flag whether to deploy or not.
222
+
- **Input:** ML model and Test dataset.
223
+
- **Output:** Performance of model and a deploy flag whether to deploy or not.
224
224
- This component compares the performance of the model with all previous deployed models on the new test dataset and decides whether to promote or not model into production. Promoting model into production happens by registering the model in AML workspace.
225
225
226
226
**Register Model**
227
227
- This component scores the model based on how accurate the predictions are in the test set.
228
-
- Input: Trained model and the deploy flag.
229
-
- Output: Registered model in Machine Learning.
228
+
- **Input:** Trained model and the deploy flag.
229
+
- **Output:** Registered model in Machine Learning.
230
230
231
-
## Deploying the Model Training Pipeline
231
+
## Deploying the model training pipeline
232
232
233
-
Next, you will deploy the model training pipeline to your new Machine Learning workspace. This pipeline will create a compute cluster instance, register a training environment defining the necessary Docker image and python packages, register a training dataset, then start the training pipeline described in the last section. When the job is complete, the trained model will be registered in the Azure ML workspace and be available for deployment.
233
+
Next, you will deploy the model training pipeline to your new machine learning workspace. This pipeline will create a compute cluster instance, register a training environment defining the necessary Docker image and python packages, register a training dataset, then start the training pipeline described in the last section. When the job is complete, the trained model will be registered in the Azure Machine Learning workspace and be available for deployment.
234
234
235
235
1. In your GitHub project repository (example: taxi-fare-regression), select**Actions**
236
236
@@ -240,57 +240,57 @@ Next, you will deploy the model training pipeline to your new Machine Learning w
> If you want to check the output of each individual step, forexample to view output of a failed run, click a job output, and then click each stepin the job to view any output of that step.
249
249
250
-
With the trained model registered in the Machine learning workspace, you are ready to deploy the model for scoring.
250
+
With the trained model registered in the machine learning workspace, you are ready to deploy the model for scoring.
251
251
252
-
### Deploying the Trained Model
252
+
### Deploying the trained model
253
253
254
-
This scenario includes prebuilt workflows fortwo approaches to deploying a trained model, batch scoring or a deploying a model to an endpoint for real-time scoring. You may run either or both of these workflows to test the performance of the modelin your Azure ML workspace.
254
+
This scenario includes prebuilt workflows fortwo approaches to deploying a trained model, batch scoring or a deploying a model to an endpoint for real-time scoring. You may run either or both of these workflows to test the performance of the modelin your Azure Machine Learning workspace.
255
255
256
256
1. In your GitHub project repository (ex: taxi-fare-regression), select**Actions**
1. Select the **deploy-online-endpoint-pipeline** from the workflows listed on the left and click **Run workflow** to execute the online endpoint deployment pipeline workflow. The steps in this pipeline will create an online endpoint in your Machine Learning workspace, create a deployment of your model to this endpoint, then allocate traffic to the endpoint.
262
+
1. Select the **deploy-online-endpoint-pipeline** from the workflows listed on the left and click **Run workflow** to execute the online endpoint deployment pipeline workflow. The steps in this pipeline will create an online endpoint in your machine learning workspace, create a deployment of your model to this endpoint, then allocate traffic to the endpoint.
1. To test this deployment, go to the **Endpoints** tab in your Machine Learning workspace, selectthe endpoint and click the **Test** Tab. You can use the sample input data located in the cloned repo at `/data/taxi-request.json` to test the endpoint.
270
+
1. To test this deployment, go to the **Endpoints** tab in your machine learning workspace, selectthe endpoint and click the **Test** Tab. You can use the sample input data located in the cloned repo at `/data/taxi-request.json` to test the endpoint.
1. Select the **deploy-batch-endpoint-pipeline** from the workflows and click **Run workflow** to execute the batch endpoint deployment pipeline workflow. The steps in this pipeline will create a new AmlCompute cluster on which to execute batch scoring, create the batch endpoint in your Machine Learning workspace, then create a deployment of your model to this endpoint.
276
+
1. Select the **deploy-batch-endpoint-pipeline** from the workflows and click **Run workflow** to execute the batch endpoint deployment pipeline workflow. The steps in this pipeline will create a new Azure Machine Learning compute cluster on which to execute batch scoring, create the batch endpoint in your machine learning workspace, then create a deployment of your model to this endpoint.
Example scenarios can be trained and deployed both forDev and Prod branches and environments. When you are satisfied with the performance of the model training pipeline, model, and deploymentin Testing, Dev pipelines and models can be replicated and deployed in the Production environment.
287
287
288
-
The sample training and deployment Machine Learning pipelines and GitHub workflows can be used as a starting point to adapt your own modeling code and data.
288
+
The sample training and deployment machine learning pipelines and GitHub workflows can be used as a starting point to adapt your own modeling code and data.
289
289
290
290
## Clean up resources
291
291
292
292
1. If you're not going to continue to use your pipeline, delete your Azure DevOps project.
293
-
1. In Azure portal, delete your resource group and Machine Learning instance.
293
+
1. In Azure portal, delete your resource group and Azure Machine Learning instance.
0 commit comments