Skip to content

Commit 7158836

Browse files
committed
editorial fixes
1 parent 694769b commit 7158836

File tree

1 file changed

+59
-75
lines changed

1 file changed

+59
-75
lines changed

articles/machine-learning/how-to-deploy-mlflow-models-online-endpoints.md

Lines changed: 59 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ For no-code-deployment, Azure Machine Learning:
3333

3434
## About the example
3535

36-
The example shows how you can deploy an MLflow model to an online endpoint to perform predictions. The example uses an MLflow model that's based on the [Diabetes dataset](https://www4.stat.ncsu.edu/~boos/var.select/diabetes.html). This dataset contains ten baseline variables: age, sex, body mass index, average blood pressure, and six blood serum measurements obtained from 442 diabetes patients. It also contains the response of interest, a quantitative measure of disease progression one year after baseline.
36+
The example shows how you can deploy an MLflow model to an online endpoint to perform predictions. The example uses an MLflow model that's based on the [Diabetes dataset](https://www4.stat.ncsu.edu/~boos/var.select/diabetes.html). This dataset contains 10 baseline variables: age, sex, body mass index, average blood pressure, and six blood serum measurements obtained from 442 diabetes patients. It also contains the response of interest, a quantitative measure of disease progression one year after baseline.
3737

3838
The model was trained using a `scikit-learn` regressor, and all the required preprocessing has been packaged as a pipeline, making this model an end-to-end pipeline that goes from raw data to predictions.
3939

@@ -56,7 +56,7 @@ Before following the steps in this article, make sure you have the following pre
5656
- Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role allowing `Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*`. For more information on roles, see [Manage access to an Azure Machine Learning workspace](how-to-assign-roles.md).
5757
- You must have an MLflow model registered in your workspace. This article registers a model trained for the [Diabetes dataset](https://www4.stat.ncsu.edu/~boos/var.select/diabetes.html) in the workspace.
5858

59-
- Additionally, you need to:
59+
- Also, you need to:
6060

6161
# [Azure CLI](#tab/cli)
6262

@@ -289,7 +289,7 @@ version = registered_model.version
289289

290290
# [Python (MLflow SDK)](#tab/mlflow)
291291

292-
We can configure the properties of this endpoint using a configuration file. In this case, we are configuring the authentication mode of the endpoint to be "key".
292+
You can configure the properties of this endpoint using a configuration file. In this case, you're configuring the authentication mode of the endpoint to be "key".
293293
294294
```python
295295
endpoint_config = {
@@ -509,7 +509,7 @@ version = registered_model.version
509509
510510
## Invoke the endpoint
511511
512-
Once your deployment is ready, you can use it to serve request. One way to test the deployment is by using the built-in invocation capability in the deployment client you are using. The following JSON is a sample request for the deployment.
512+
Once your deployment is ready, you can use it to serve request. One way to test the deployment is by using the built-in invocation capability in the deployment client you're using. The following JSON is a sample request for the deployment.
513513

514514
**sample-request-sklearn.json**
515515

@@ -536,7 +536,7 @@ ml_client.online_endpoints.invoke(
536536
# [Python (MLflow SDK)](#tab/mlflow)
537537

538538
```python
539-
# Read the sample request we have in the json file to construct a pandas data frame
539+
# Read the sample request that's in the json file to construct a pandas data frame
540540
with open("sample-request-sklearn.json", "r") as f:
541541
sample_request = json.loads(f.read())
542542
samples = pd.DataFrame(**sample_request["input_data"])
@@ -571,59 +571,59 @@ The response will be similar to the following text:
571571

572572
## Customize MLflow model deployments
573573

574-
MLflow models can be deployed to online endpoints without indicating a scoring script in the deployment definition. However, you can opt to customize how inference is executed.
574+
You don't have to specify a scoring script in the deployment definition of an MLflow model to an online endpoint. However, you can opt to do so and customize how inference gets executed.
575575
576-
You will typically select this workflow when:
576+
You'll typically want to customize your MLflow model deployment when:
577577

578578
> [!div class="checklist"]
579579
> - The model doesn't have a `PyFunc` flavor on it.
580-
> - You need to customize the way the model is run, for instance, use an specific flavor to load it with `mlflow.<flavor>.load_model()`.
581-
> - You need to do pre/post processing in your scoring routine when it is not done by the model itself.
582-
> - The output of the model can't be nicely represented in tabular data. For instance, it is a tensor representing an image.
580+
> - You need to customize the way the model is run, for instance, to use a specific flavor to load the model, using `mlflow.<flavor>.load_model()`.
581+
> - You need to do pre/post processing in your scoring routine when it's not done by the model itself.
582+
> - The output of the model can't be nicely represented in tabular data. For instance, it's a tensor representing an image.
583583

584584
> [!IMPORTANT]
585-
> If you choose to indicate an scoring script for an MLflow model deployment, you will also have to specify the environment where the deployment will run.
585+
> If you choose to specify a scoring script for an MLflow model deployment, you'll also have to specify the environment where the deployment will run.
586586
587587
### Steps
588588
589-
Use the following steps to deploy an MLflow model with a custom scoring script.
589+
To deploy an MLflow model with a custom scoring script:
590590
591-
1. Identify the folder where your MLflow model is placed.
591+
1. Identify the folder where your MLflow model is located.
592592
593593
a. Go to the [Azure Machine Learning studio](https://ml.azure.com).
594594
595-
b. Go to the section __Models__.
595+
b. Go to the __Models__ section.
596596
597597
c. Select the model you're trying to deploy and go to its __Artifacts__ tab.
598598

599-
d. Take note of the folder that is displayed. This folder was indicated when the model was registered.
599+
d. Take note of the folder that is displayed. This folder was specified when the model was registered.
600600

601601
:::image type="content" source="media/how-to-deploy-mlflow-models-online-endpoints/mlflow-model-folder-name.png" lightbox="media/how-to-deploy-mlflow-models-online-endpoints/mlflow-model-folder-name.png" alt-text="Screenshot showing the folder where the model artifacts are placed.":::
602602

603-
1. Create a scoring script. Notice how the folder name `model` you identified before has been included in the `init()` function.
603+
1. Create a scoring script. Notice how the folder name `model` that you previously identified is included in the `init()` function.
604+
605+
> [!TIP]
606+
> The following scoring script is provided as an example about how to perform inference with an MLflow model. You can adapt this script to your needs or change any of its parts to reflect your scenario.
604607
605608
__score.py__
606609
607610
:::code language="python" source="~/azureml-examples-main/cli/endpoints/online/ncd/sklearn-diabetes/src/score.py" highlight="14":::
608611
609-
> [!TIP]
610-
> The previous scoring script is provided as an example about how to perform inference of an MLflow model. You can adapt this example to your needs or change any of its parts to reflect your scenario.
611-
612612
> [!WARNING]
613-
> __MLflow 2.0 advisory__: The provided scoring script will work with both MLflow 1.X and MLflow 2.X. However, be advised that the expected input/output formats on those versions may vary. Check the environment definition used to ensure you are using the expected MLflow version. Notice that MLflow 2.0 is only supported in Python 3.8+.
613+
> __MLflow 2.0 advisory__: The provided scoring script will work with both MLflow 1.X and MLflow 2.X. However, be advised that the expected input/output formats on those versions might vary. Check the environment definition used to ensure you're using the expected MLflow version. Notice that MLflow 2.0 is only supported in Python 3.8+.
614614
615-
1. Let's create an environment where the scoring script can be executed. Since our model is MLflow, the conda requirements are also specified in the model package (for more details about MLflow models and the files included on it see The MLmodel format). We are going then to build the environment using the conda dependencies from the file. However, we need also to include the package `azureml-inference-server-http` which is required for Online Deployments in Azure Machine Learning.
615+
1. Create an environment where the scoring script can be executed. Since the model is an MLflow model, the conda requirements are also specified in the model package. For more details about the files included in an MLflow model see [The MLmodel format](concept-mlflow-models.md#the-mlmodel-format). You'll then build the environment using the conda dependencies from the file. However, you need to also include the package `azureml-inference-server-http`, which is required for online deployments in Azure Machine Learning.
616616
617-
The conda definition file looks as follows:
617+
The conda definition file is as follows:
618618
619619
__conda.yml__
620620
621621
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/ncd/sklearn-diabetes/environment/conda.yaml":::
622622
623623
> [!NOTE]
624-
> Note how the package `azureml-inference-server-http` has been added to the original conda dependencies file.
624+
> The `azureml-inference-server-http` package has been added to the original conda dependencies file.
625625
626-
We will use this conda dependencies file to create the environment:
626+
You'll use this conda dependencies file to create the environment:
627627
628628
# [Azure CLI](#tab/cli)
629629
@@ -640,32 +640,27 @@ Use the following steps to deploy an MLflow model with a custom scoring script.
640640
641641
# [Python (MLflow SDK)](#tab/mlflow)
642642
643-
*This operation is not supported in MLflow SDK*
643+
*This operation isn't supported in MLflow SDK*
644644
645645
# [Studio](#tab/studio)
646-
647-
In the [studio](https://ml.azure.com), follow these steps:
648-
649-
1. Navigate to the __Environments__ tab on the side menu.
646+
647+
1. Go to the __Environments__ tab on the side menu.
650648
1. Select the tab __Custom environments__ > __Create__.
651649
1. Enter the name of the environment, in this case `sklearn-mlflow-online-py37`.
652-
1. On __Select environment source__ select __Use existing docker image with optional conda file__.
653-
1. On __Container registry image path__, enter `mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04`.
654-
1. On __Customize__ section copy the content of the file `sklearn-diabetes/environment/conda.yml` we introduced before.
655-
1. Click on __Next__ to go tp the "Tags" page, and then __Next__ again.
656-
1. On the "Review" page, select __Create__.
657-
1. The environment is ready to be used.
658-
650+
1. For __Select environment source__, choose __Use existing docker image with optional conda file__.
651+
1. For __Container registry image path__, enter `mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04`.
652+
1. Select __Next__ to go to the __Customize__ section.
653+
1. Copy the content of the `sklearn-diabetes/environment/conda.yml` file and paste it in the text box.
654+
1. Select __Next__ to go to the __Tags__ page, and then __Next__ again.
655+
1. On the __Review__ page, select __Create__. The environment is ready for use.
659656
---
660657
661-
1. Let's create the deployment now:
658+
1. Create the deployment:
662659
663660
# [Azure CLI](#tab/cli)
664-
665-
Create a deployment configuration file:
666-
667-
__deployment.yml__
668-
661+
662+
Create a deployment configuration file __deployment.yml__:
663+
669664
```yaml
670665
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
671666
name: sklearn-diabetes-custom
@@ -680,9 +675,9 @@ Use the following steps to deploy an MLflow model with a custom scoring script.
680675
instance_type: Standard_F2s_v2
681676
instance_count: 1
682677
```
683-
678+
684679
Create the deployment:
685-
680+
686681
```azurecli
687682
az ml online-deployment create -f deployment.yml
688683
```
@@ -706,39 +701,37 @@ Use the following steps to deploy an MLflow model with a custom scoring script.
706701
707702
# [Python (MLflow SDK)](#tab/mlflow)
708703
709-
*This operation is not supported in MLflow SDK*
704+
*This operation isn't supported in MLflow SDK*
710705
711706
# [Studio](#tab/studio)
712-
713-
In the [Azure Machine Learning studio](https://ml.azure.com), follow these steps:
714-
707+
715708
1. From the __Endpoints__ page, Select **+Create**.
716709
1. Select the MLflow model you registered previously.
717710
1. Select __More options__ in the endpoint creation wizard to open up advanced options.
718-
711+
719712
:::image type="content" source="media/how-to-deploy-mlflow-models-online-endpoints/select-advanced-deployment-options.png" lightbox="media/how-to-deploy-mlflow-models-online-endpoints/select-advanced-deployment-options.png" alt-text="Screenshot showing how to select advanced deployment options when creating an endpoint":::
720-
713+
721714
1. Provide a name and authentication type for the endpoint, and then select __Next__ to see that the model you selected is being used for your deployment.
722-
1. Select __Next__ to continue to the "Deployment" page.
723-
1. Select __Next__ to go to the "Code + environment" page. When you select a model registered in MLflow format, you don't need to specify a scoring script or an environment on this page. However, you want to specify one in this section
715+
1. Select __Next__ to continue to the ___Deployment__ page.
716+
1. Select __Next__ to go to the __Code + environment__ page. When you select a model registered in MLflow format, you don't need to specify a scoring script or an environment on this page. However, you want to specify one in this section
724717
1. Select the slider next to __Customize environment and scoring script__.
725718
726719
:::image type="content" source="media/how-to-deploy-mlflow-models-online-endpoints/configure-scoring-script-mlflow.png" lightbox="media/how-to-deploy-mlflow-models-online-endpoints/configure-scoring-script-mlflow.png" alt-text="Screenshot showing how to indicate an environment and scoring script for MLflow models":::
727-
728-
1. Browse to select the scoring script you created before.
720+
721+
1. Browse to select the scoring script you created previously.
729722
1. Select __Custom environments__ for the environment type.
730-
1. Select the custom environment you created before and select __Next__.
723+
1. Select the custom environment you created previously, and select __Next__.
731724
1. Complete the wizard to deploy the model to the endpoint.
732725
733726
---
734727
735-
1. Once your deployment completes, your deployment is ready to serve request. One of the easier ways to test the deployment is by using a sample request file along with the `invoke` method.
728+
1. Once your deployment completes, it is ready to serve requests. One way to test the deployment is by using a sample request file along with the `invoke` method.
736729
737730
**sample-request-sklearn.json**
738731
739732
:::code language="json" source="~/azureml-examples-main/cli/endpoints/online/ncd/sample-request-sklearn.json":::
740733
741-
To submit a request to the endpoint, you can do as follows:
734+
Submit a request to the endpoint as follows:
742735
743736
# [Azure CLI](#tab/cli)
744737
@@ -756,18 +749,16 @@ Use the following steps to deploy an MLflow model with a custom scoring script.
756749
757750
# [Python (MLflow SDK)](#tab/mlflow)
758751
759-
*This operation is not supported in MLflow SDK*
752+
*This operation isn't supported in MLflow SDK*
760753
761754
# [Studio](#tab/studio)
762-
763-
MLflow models can use the __Test__ tab to create invocations to the created endpoints. To do that:
764-
755+
765756
1. Go to the __Endpoints__ tab and select the new endpoint created.
766757
1. Go to the __Test__ tab.
767-
1. Paste the content of the file `sample-request-sklearn.json` into the __Input data to test endpoint__ box.
758+
1. Paste the content of the `sample-request-sklearn.json` file into the __Input data to test endpoint__ box.
768759
1. Select __Test__.
769760
1. The predictions will show up under "Test results" to the right-hand side of the box.
770-
761+
771762
---
772763
773764
The response will be similar to the following text:
@@ -782,19 +773,19 @@ Use the following steps to deploy an MLflow model with a custom scoring script.
782773
```
783774
784775
> [!WARNING]
785-
> __MLflow 2.0 advisory__: In MLflow 1.X, the key `predictions` will be missing.
776+
> __MLflow 2.0 advisory__: In MLflow 1.X, the `predictions` key will be missing.
786777
787778
788779
## Clean up resources
789780
790-
Once you're done with the endpoint, you can delete the associated resources:
781+
Once you're done using the endpoint, delete its associated resources:
791782
792783
# [Azure CLI](#tab/cli)
793784
794785
:::code language="azurecli" source="~/azureml-examples-main/cli/deploy-managed-online-endpoint-ncd.sh" ID="delete_endpoint":::
795786
796787
# [Python (Azure Machine Learning SDK)](#tab/sdk)
797-
788+
798789
```python
799790
ml_client.online_endpoints.begin_delete(endpoint_name)
800791
```
@@ -815,15 +806,8 @@ deployment_client.delete_endpoint(endpoint_name)
815806
816807
---
817808
818-
## Next steps
819-
820-
To learn more, review these articles:
809+
## Related content
821810
822811
- [Deploy models with REST](how-to-deploy-with-rest.md)
823-
- [Create and use online endpoints in the studio](how-to-use-managed-online-endpoint-studio.md)
824812
- [Safe rollout for online endpoints](how-to-safely-rollout-online-endpoints.md)
825-
- [How to autoscale managed online endpoints](how-to-autoscale-endpoints.md)
826-
- [Use batch endpoints for batch scoring](batch-inference/how-to-use-batch-endpoint.md)
827-
- [View costs for an Azure Machine Learning managed online endpoint](how-to-view-online-endpoints-costs.md)
828-
- [Access Azure resources with an online endpoint and managed identity](how-to-access-resources-from-endpoints-managed-identities.md)
829-
- [Troubleshoot online endpoint deployment](how-to-troubleshoot-managed-online-endpoints.md)
813+
- [Troubleshoot online endpoint deployment](how-to-troubleshoot-managed-online-endpoints.md)- [How to autoscale managed online endpoints](how-to-autoscale-endpoints.md)

0 commit comments

Comments
 (0)