Merge pull request #226058 from Blackmist/mlops-update

Mlops update

Author: JamesJBarnett

articles/machine-learning/how-to-setup-mlops-azureml.md

@@ -17,15 +17,15 @@ ms.custom: cli-v2, sdk-v2
 
 [!INCLUDE [dev v2](../../includes/machine-learning-dev-v2.md)]
 
-Azure Machine Learning allows you to integration with [Azure DevOps pipeline](/azure/devops/pipelines/) to automate the machine learning lifecycle. Some of the operations you can automate are:
+Azure Machine Learning allows you to integrate with [Azure DevOps pipeline](/azure/devops/pipelines/) to automate the machine learning lifecycle. Some of the operations you can automate are:
 
 * Deployment of AzureML infrastructure
 * Data preparation (extract, transform, load operations)
 * Training machine learning models with on-demand scale-out and scale-up
 * Deployment of machine learning models as public or private web services
 * Monitoring deployed machine learning models (such as for performance analysis)
 
-In this article, you learn about using Azure Machine Learning to set up an end-to-end MLOps pipeline that runs a linear regression to predict taxi fares in NYC. The pipeline is made up of components, each serving  different functions, which can be registered with the workspace, versioned, and reused with various inputs and outputs. you are going to be using the [recommended Azure architecture for MLOps](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) and [Azure MLOps (v2) solution accelerator](https://github.com/Azure/mlops-v2) to quickly set up an MLOps project in AzureML.
+In this article, you learn about using Azure Machine Learning to set up an end-to-end MLOps pipeline that runs a linear regression to predict taxi fares in NYC. The pipeline is made up of components, each serving  different functions, which can be registered with the workspace, versioned, and reused with various inputs and outputs. you're going to be using the [recommended Azure architecture for MLOps](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) and [Azure MLOps (v2) solution accelerator](https://github.com/Azure/mlops-v2) to quickly setup an MLOps project in AzureML.
 
 > [!TIP]
 > We recommend you understand some of the [recommended Azure architectures](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) for MLOps before implementing any solution. You'll need to pick the best architecture for your given Machine learning project.
@@ -34,12 +34,10 @@ In this article, you learn about using Azure Machine Learning to set up an end-t
 
 - An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/).
 - An Azure Machine Learning workspace.
-- The Azure Machine Learning [SDK v2 for Python](https://aka.ms/sdk-v2-install).
-- The Azure Machine Learning [CLI v2](how-to-configure-cli.md).
 - Git running on your local machine.
 - An [organization](/azure/devops/organizations/accounts/create-organization) in Azure DevOps.
 - [Azure DevOps project](how-to-devops-machine-learning.md) that will host the source repositories and pipelines.
-- The [Terraform extension for Azure DevOps](https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks) if you are using Azure DevOps + Terraform to spin up infrastructure
+- The [Terraform extension for Azure DevOps](https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks) if you're using Azure DevOps + Terraform to spin up infrastructure
 
 > [!NOTE]
 >
@@ -66,7 +64,7 @@ Before you can set up an MLOps project with AzureML, you need to set up authenti
 
     ![Screenshot of the cloud shell environment dropdown.](./media/how-to-setup-mlops-azureml/PS_CLI1_1.png)
 
-1. Copy the bash commands below to your computer and update the **projectName**, **subscriptionId**, and **environment** variables with the values for your project. If you are creating both a Dev and Prod environment, you'll need to run this script once for each environment, creating a service principal for each. This command will also grant the **Contributor** role to the service principal in the subscription provided. This is required for Azure DevOps to properly use resources in that subscription. 
+1. Copy the following bash commands to your computer and update the **projectName**, **subscriptionId**, and **environment** variables with the values for your project. If you're creating both a Dev and Prod environment, you'll need to run this script once for each environment, creating a service principal for each. This command will also grant the **Contributor** role to the service principal in the subscription provided. This is required for Azure DevOps to properly use resources in that subscription. 
 
     ``` bash
     projectName="<your project name>"
@@ -94,7 +92,7 @@ Before you can set up an MLOps project with AzureML, you need to set up authenti
     }
     ```
 
-1. Repeat **Step 3.** if you're creating service principals for Dev and Prod environments.
+1. Repeat **Step 3.** if you're creating service principals for Dev and Prod environments. For this demo, we'll be creating only one environment, which is Prod.
 
 1. Close the Cloud Shell once the service principals are created. 
       
@@ -146,9 +144,9 @@ Before you can set up an MLOps project with AzureML, you need to set up authenti
      - **Tenant ID** - Use the `tenant` from **Step 1.** output as the Tenant ID
 
 
-6. Name the service connection **Azure-ARM-Dev**.  
+6. Name the service connection **Azure-ARM-Prod**.  
  
-7. Select **Grant access permission to all pipelines**, then select **Verify and Save**. Repeat this step to create another service connection **Azure-ARM-Prod** using the details of the Prod service principal created in **Step 1.**
+7. Select **Grant access permission to all pipelines**, then select **Verify and Save**. 
 
 The Azure DevOps setup is successfully finished.
 
@@ -158,32 +156,20 @@ The Azure DevOps setup is successfully finished.
    
 1. Open the Repos section and select **Import Repository**
 
-    ![Screenshot of ADO import repo first time.](./media/how-to-setup-mlops-azureml/import_repo_first_time.png)
+    ![Screenshot of Azure DevOps import repo first time.](./media/how-to-setup-mlops-azureml/import_repo_first_time.png)
 
-1. Enter https://github.com/Azure/mlops-v2-ado-demo into the Clone URL field. Click import at the bottom of the page
+1. Enter https://github.com/Azure/mlops-v2-ado-demo into the Clone URL field. Select import at the bottom of the page
 
-    ![Screenshot of ADO import MLOps demo repo.](./media/how-to-setup-mlops-azureml/import_repo_Git_template.png)
-
-
-1. Open the Repos section. Click on the default repo name at the top of the screen and select Import Repository
-
-    ![Screenshot of ADO import repo.](./media/how-to-setup-mlops-azureml/ado-import-repo.png)
-
-1. Enter https://github.com/Azure/mlops-templates into the Clone URL field. Click import at the bottom of the page
-
-    ![Screenshot of ADO import MLOps template repo.](./media/how-to-setup-mlops-azureml/ado-import-mlops-templates.png)
-
-    > [!TIP]
-    > Learn more about the MLOps v2 accelerator structure and the MLOps [template](https://github.com/Azure/mlops-v2/)
+    ![Screenshot of Azure DevOps import MLOps demo repo.](./media/how-to-setup-mlops-azureml/import_repo_Git_template.png)
 
 1. Open the **Project settings** at the bottom of the left hand navigation pane
 
-1.  Under the Repos section, click **Repositories**. Select the repository you created in **Step 6.** Select the **Security** tab
+1.  Under the Repos section, select **Repositories**. Select the repository you created in **Step 6.** Select the **Security** tab
 
 1. Under the User permissions section, select the **mlopsv2 Build Service** user. Change the permission **Contribute** permission to **Allow** and the **Create branch** permission to **Allow**.
-   ![Screenshot of ADO permissions.](./media/how-to-setup-mlops-azureml/ado-permissions-repo.png)
+   ![Screenshot of Azure DevOps permissions.](./media/how-to-setup-mlops-azureml/ado-permissions-repo.png)
 
-1. Open the **Pipelines** section in the left hand navigation pane and click on the 3 vertical dots next to the **Create Pipelines** button. Select **Manage Security**
+1. Open the **Pipelines** section in the left hand navigation pane and select on the 3 vertical dots next to the **Create Pipelines** button. Select **Manage Security**
 
    ![Screenshot of Pipeline security.](./media/how-to-setup-mlops-azureml/ado-open-pipelinesSecurity.png)
 
@@ -202,7 +188,10 @@ This step deploys the training pipeline to the Azure Machine Learning workspace
 > Make sure you understand the [Architectural Patterns](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2) of the solution accelerator before you checkout the MLOps v2 repo and deploy the infrastructure. In examples you'll use the [classical ML project type](/azure/architecture/data-guide/technology-choices/machine-learning-operations-v2#classical-machine-learning-architecture).
 
 ### Run Azure infrastructure pipeline
-1. Go to the first repo you imported in the previous section, `mlops-v2-ado-demo`. Make sure you have the `main` branch selected and then select the **config-infra-dev.yml** file.
+1. Go to your repository, `mlops-v2-ado-demo`, and select the **config-infra-prod.yml** file.
+
+    > [!IMPORTANT]
+    > Make sure you've selected the **main** branch of the repo.
    
    ![Screenshot of Repo in ADO.](./media/how-to-setup-mlops-azureml/ADO-repo.png)
    
@@ -216,9 +205,7 @@ This step deploys the training pipeline to the Azure Machine Learning workspace
    > [!NOTE]
    > If you are running a Deep Learning workload such as CV or NLP, ensure your GPU compute is available in your deployment zone.
 
-1. Click Commit and push code to get these values into the pipeline. 
-
-1. Repeat this step for **config-infra-prod.yml** file.
+1. Select Commit and push code to get these values into the pipeline. 
 
 1. Go to Pipelines section 
    
@@ -236,15 +223,10 @@ This step deploys the training pipeline to the Azure Machine Learning workspace
 
 1. Select **Existing Azure Pipeline YAML File**
 
-   ![Screenshot of ADO Pipeline page on configure step.](./media/how-to-setup-mlops-azureml/ADO-configure-pipelines.png)
+   ![Screenshot of Azure DevOps Pipeline page on configure step.](./media/how-to-setup-mlops-azureml/ADO-configure-pipelines.png)
 
 
-1. Select `main` as a branch and choose based on your deployment method your preferred yml path. 
-    - For a terraform scenario, choose `infrastructure/pipelines/tf-ado-deploy-infra.yml`, then select **Continue**. 
-    - For a bicep scenario, choose `infrastructure/pipelines/bicep-ado-deploy-infra.yml`, then select **Continue**.
-
-> [!CAUTION]
-> For this example, make sure the [Terraform extension for Azure DevOps](https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks) is installed.
+1. Select the `main` branch and choose `mlops/devops-pipelines/cli-ado-deploy-infra.yml`, then select **Continue**. 
 
 1. Run the pipeline; it will take a few minutes to finish. The pipeline should create the following artifacts:
    * Resource Group for your Workspace including Storage Account, Container Registry, Application Insights, Keyvault and the Azure Machine Learning Workspace itself.
@@ -323,19 +305,13 @@ This step deploys the training pipeline to the Azure Machine Learning workspace
 
 1. Select **Existing Azure Pipeline YAML File**
 
-   ![Screenshot of ADO Pipeline page on configure step.](./media/how-to-setup-mlops-azureml/ADO-configure-pipelines.png)
+   ![Screenshot of Azure DevOps Pipeline page on configure step.](./media/how-to-setup-mlops-azureml/ADO-configure-pipelines.png)
 
-1. Select `main` as a branch and choose:
-        
-      - For Managed Batch Endpoint `/mlops/devops-pipelines/deploy-batch-endpoint-pipeline.yml`
-      
-      - For Managed Online Endpoint `/mlops/devops-pipelines/deploy-online-endpoint-pipeline.yml`
-      
-   Then select **Continue**.  
+1. Select `main` as a branch and choose Managed Online Endpoint `/mlops/devops-pipelines/deploy-online-endpoint-pipeline.yml` then select **Continue**.  
 
-1. Batch/Online endpoint names need to be unique, so change **[your endpoint-name]** to another unique name and then select **Run**.
+1. Online endpoint names need to be unique, so change `taxi-online-$(namespace)$(postfix)$(environment)` to another unique name and then select **Run**. No need to change the default if it doesn't fail.
 
-   ![Screenshot of ADO batch deploy script.](./media/how-to-setup-mlops-azureml/ADO-batch-pipeline.png)
+   ![Screenshot of Azure DevOps batch deploy script.](./media/how-to-setup-mlops-azureml/ADO-batch-pipeline.png)
    
 > [!IMPORTANT]
 > If the run fails due to an existing online endpoint name, recreate the pipeline as described previously and change **[your endpoint-name]** to **[your endpoint-name (random number)]**