You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-deploy-online-endpoints.md
+58-17Lines changed: 58 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -53,7 +53,7 @@ Before following the steps in this article, make sure you have the following pre
53
53
54
54
* An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/).
55
55
56
-
* An Azure Machine Learning workspace and a compute instance. If you don't have these resources and want to create them, see [Create resources you need to get started](quickstart-create-resources.md).
56
+
* An Azure Machine Learning workspace and a compute instance. If you don't have these resources, see [Create resources you need to get started](quickstart-create-resources.md).
57
57
58
58
* Azure role-based access control (Azure RBAC) is used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the _owner_ or _contributor_ role for the Azure Machine Learning workspace, or a custom role allowing `Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*`. For more information, see [Manage access to an Azure Machine Learning workspace](how-to-assign-roles.md).
59
59
@@ -68,7 +68,7 @@ Before following the steps in this article, make sure you have the following pre
68
68
69
69
---
70
70
71
-
* Ensure that you have enough virtual machine (VM) quota allocated for deployment. Azure Machine Learning reserves 20% of your compute resources for performing upgrades on some VM SKUs. For example, if you request 10 instances in a deployment, you must have a quota for 12 for each number of cores for the VM SKU. Failure to account for the extra compute resources results in an error. There are some VM SKUs that are exempt from the extra quota reservation. For more information on quota allocation, see [virtual machine quota allocation for deployment](how-to-manage-quotas.md#virtual-machine-quota-allocation-for-deployment).
71
+
* Ensure that you have enough virtual machine (VM) quota allocated for deployment. Azure Machine Learning reserves 20% of your compute resources for performing upgrades on some VM SKUs. For example, if you request 10 instances in a deployment, you must have a quota of 12 for each number of cores for the VM SKU. Failure to account for the extra compute resources results in an error. There are some VM SKUs that are exempt from the extra quota reservation. For more information on quota allocation, see [virtual machine quota allocation for deployment](how-to-manage-quotas.md#virtual-machine-quota-allocation-for-deployment).
72
72
73
73
* Alternatively, you could use quota from Azure Machine Learning's shared quota pool for a limited time. [!INCLUDE [machine-learning-shared-quota](includes/machine-learning-shared-quota.md)]
74
74
@@ -87,7 +87,7 @@ If you haven't already set the defaults for the Azure CLI, save your default set
87
87
88
88
### Clone the examples repository
89
89
90
-
To follow along with this article, first clone the [azureml-examples repository](https://github.com/azure/azureml-examples), then change into the repository's *azureml-examples/cli*` directory:
90
+
To follow along with this article, first clone the [azureml-examples repository](https://github.com/azure/azureml-examples), then change into the repository's *azureml-examples/cli* directory:
@@ -179,7 +179,9 @@ cd azureml-examples/cli/endpoints/online/model-1
179
179
If you cloned the examples repo, your local machine already has copies of the files for this example, and you can skip to the next section. If you didn't clone the repo, you can download it to your local machine.
180
180
181
181
1. Go to [https://github.com/Azure/azureml-examples/](https://github.com/Azure/azureml-examples/).
182
+
182
183
1. Go to the **<> Code** button on the page, and then select **Download ZIP** from the **Local** tab.
184
+
183
185
1. Locate the folder */cli/endpoints/online/model-1/model* and the file */cli/endpoints/online/model-1/onlinescoring/score.py*.
184
186
185
187
# [ARM template](#tab/arm)
@@ -258,15 +260,15 @@ First define the name of the online endpoint, then configure the endpoint.
258
260
Replace `<YOUR_ENDPOINT_NAME>` with a name that's unique in the Azure region, or use the example method to define a random name. Be sure to delete the method that you don't use. For more information on the naming rules, see [endpoint limits](how-to-manage-quotas.md#azure-machine-learning-online-endpoints-and-batch-endpoints).
@@ -692,31 +694,43 @@ A model registration is a logical entity in the workspace that can contain a sin
692
694
To register the example model, follow these steps:
693
695
694
696
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
695
-
1. In the left pane, select the **Models** page.
697
+
698
+
1. In the side pane, select the **Models** page.
699
+
696
700
1. Select **Register**, and then choose **From local files**.
701
+
697
702
1. Select __Unspecified type__ for the __Model type__.
703
+
698
704
1. Select __Browse__, and choose __Browse folder__.
699
705
700
706
:::image type="content" source="media/how-to-deploy-online-endpoints/register-model-folder.png" alt-text="A screenshot of the browse folder option." lightbox="media/how-to-deploy-online-endpoints/register-model-folder.png":::
701
707
702
708
1. Select the `\azureml-examples\cli\endpoints\online\model-1\model` folder from the local copy of the repo you cloned or downloaded earlier. When prompted, select __Upload__ and wait for the upload to complete.
709
+
703
710
1. Select __Next__ after the folder upload is completed.
711
+
704
712
1. Enter a friendly __Name__ for the model. The steps in this article assume the model is named `model-1`.
713
+
705
714
1. Select __Next__, and then __Register__ to complete registration.
706
715
707
716
For more information on working with registered models, see [Work with registered models](how-to-manage-models.md).
708
717
709
718
### Create and register the environment
710
719
711
720
1. In the side navigation bar, select the **Environments** page.
721
+
712
722
1. Select the **Custom environments** tab, then choose **Create**.
723
+
713
724
1. On the **Settings** page, provide a name, such as *my-env* for the environment.
725
+
714
726
1. For **Select environment source**, choose **Use existing docker image with optional conda source**.
715
727
716
728
:::image type="content" source="media/how-to-deploy-online-endpoints/create-environment.png" alt-text="A screenshot showing how to create a custom environment." lightbox="media/how-to-deploy-online-endpoints/create-environment.png":::
717
729
718
730
1. Select **Next** to go to the **Customize** page.
731
+
719
732
1. Copy the contents of the *\azureml-examples\cli\endpoints\online\model-1\environment\conda.yaml* file from the repo you cloned or downloaded earlier.
733
+
720
734
1. Paste the contents into the text box.
721
735
722
736
:::image type="content" source="media/how-to-deploy-online-endpoints/customize-environment-with-conda-file.png" alt-text="A screenshot showing how to customize the environment, using a conda file." lightbox="media/how-to-deploy-online-endpoints/customize-environment-with-conda-file.png":::
@@ -754,7 +768,7 @@ Your deployment configuration uses the registered model that you wish to deploy
754
768
755
769
# [Azure CLI](#tab/cli)
756
770
757
-
Use the registered assets (model and environment) in your deployment definition. The following snippet shows the `endpoints/online/managed/sample/blue-deployment-with-registered-assets.yml` file, with all the required inputs to configure a deployment:
771
+
Use the registered assets (model and environment) in your deployment definition. The following snippet shows the *endpoints/online/managed/sample/blue-deployment-with-registered-assets.yml* file, with all the required inputs to configure a deployment:
758
772
759
773
__blue-deployment-with-registered-assets.yml__
760
774
@@ -889,20 +903,27 @@ Use the studio to create a managed online endpoint directly in your browser. Whe
889
903
One way to create a managed online endpoint in the studio is from the **Models** page. This method also provides an easy way to add a model to an existing managed online deployment. To deploy the model named `model-1` that you registered previously in the [Register your model and environment](#register-your-model-and-environment) section:
890
904
891
905
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
906
+
892
907
1. In the side navigation bar, select the **Models** page.
908
+
893
909
1. Select the model named `model-1` by checking the circle next to its name.
910
+
894
911
1. Select **Deploy** > **Real-time endpoint**.
895
912
896
913
:::image type="content" source="media/how-to-deploy-online-endpoints/deploy-from-models-page.png" lightbox="media/how-to-deploy-online-endpoints/deploy-from-models-page.png" alt-text="A screenshot of creating a managed online endpoint from the Models UI.":::
897
-
914
+
898
915
This action opens up a window where you can specify details about your endpoint.
899
916
900
917
:::image type="content" source="media/how-to-deploy-online-endpoints/online-endpoint-wizard.png" lightbox="media/how-to-deploy-online-endpoints/online-endpoint-wizard.png" alt-text="A screenshot of a managed online endpoint create wizard.":::
901
918
902
919
1. Enter an __Endpoint name__ that's unique in the Azure region. For more information on the naming rules, see [endpoint limits](how-to-manage-quotas.md#azure-machine-learning-online-endpoints-and-batch-endpoints).
920
+
903
921
1. Keep the default selection: __Managed__ for the compute type.
922
+
904
923
1. Keep the default selection: __key-based authentication__ for the authentication type. For more information on authenticating, see [Authenticate clients for online endpoints](how-to-authenticate-online-endpoint.md).
924
+
905
925
1. Select __Next__, until you get to the **Deployment** page. Here, toggle __Application Insights diagnostics__ to *Enabled* to allow you to view graphs of your endpoint's activities in the studio later and analyze metrics and logs using Application Insights.
926
+
906
927
1. Select __Next__ to go to the **Code + environment** page. Here, select the following options:
907
928
908
929
* __Select a scoring script for inferencing__: Browse and select the *\azureml-examples\cli\endpoints\online\model-1\onlinescoring\score.py* file from the repo you cloned or downloaded earlier.
@@ -911,12 +932,15 @@ One way to create a managed online endpoint in the studio is from the **Models**
911
932
:::image type="content" source="media/how-to-deploy-online-endpoints/deploy-with-custom-environment.png" lightbox="media/how-to-deploy-online-endpoints/deploy-with-custom-environment.png" alt-text="A screenshot showing selection of a custom environment for deployment.":::
912
933
913
934
1. Select __Next__, accepting defaults, until you're prompted to create the deployment.
935
+
914
936
1. Review your deployment settings and select the __Create__ button.
915
937
916
938
Alternatively, you can create a managed online endpoint from the **Endpoints** page in the studio.
917
939
918
940
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
919
-
1. In the left pane, select the **Endpoints** page.
941
+
942
+
1. In the side pane, select the **Endpoints** page.
943
+
920
944
1. Select **+ Create**.
921
945
922
946
:::image type="content" source="media/how-to-deploy-online-endpoints/endpoint-create-managed-online-endpoint.png" lightbox="media/how-to-deploy-online-endpoints/endpoint-create-managed-online-endpoint.png" alt-text="A screenshot for creating managed online endpoint from the Endpoints tab.":::
@@ -937,7 +961,7 @@ This action opens up a window for you to select your model and specify details a
937
961
938
962
To debug errors in your deployment, see [Troubleshooting online endpoint deployments](./how-to-troubleshoot-online-endpoints.md).
939
963
940
-
### Check the status of the endpoint
964
+
### Check the status of the online endpoint
941
965
942
966
# [Azure CLI](#tab/cli)
943
967
@@ -982,7 +1006,9 @@ To debug errors in your deployment, see [Troubleshooting online endpoint deploym
982
1006
You can view all your managed online endpoints in the **Endpoints** page. Go to the endpoint's **Details** page to find critical information including the endpoint URI, status, testing tools, activity monitors, deployment logs, and sample consumption code:
983
1007
984
1008
1. In the side navigation bar, select **Endpoints**. Here, you can see a list of all the endpoints in the workspace.
1009
+
985
1010
1. (Optional) Create a **Filter** on **Compute type** to show only **Managed** compute types.
1011
+
986
1012
1. Select an endpoint name to view the endpoint's __Details__ page.
987
1013
988
1014
:::image type="content" source="media/how-to-deploy-online-endpoints/managed-endpoint-details-page.png" lightbox="media/how-to-deploy-online-endpoints/managed-endpoint-details-page.png" alt-text="Screenshot of managed endpoint details view.":::
@@ -1102,8 +1128,11 @@ Using the `MLClient` created earlier, get a handle to the endpoint. The endpoint
1102
1128
Use the **Test** tab in the endpoint's details page to test your managed online deployment. Enter sample input and view the results.
1103
1129
1104
1130
1. Select the **Test** tab in the endpoint's detail page.
1131
+
1105
1132
1. Use the dropdown to select the deployment you want to test.
1133
+
1106
1134
1. Enter the [sample input](https://github.com/Azure/azureml-examples/blob/main/sdk/python/endpoints/online/model-1/sample-request.json).
1135
+
1107
1136
1. Select **Test**.
1108
1137
1109
1138
:::image type="content" source="media/how-to-deploy-online-endpoints/test-deployment.png" lightbox="media/how-to-deploy-online-endpoints/test-deployment.png" alt-text="A screenshot of testing a deployment by providing sample data, directly in your browser.":::
@@ -1133,8 +1162,11 @@ If you want to update the code, model, or environment, update the YAML file, and
1133
1162
To understand how `update` works:
1134
1163
1135
1164
1. Open the file *online/model-1/onlinescoring/score.py*.
1165
+
1136
1166
1. Change the last line of the `init()` function: After `logging.info("Init complete")`, add `logging.info("Updated successfully")`.
1167
+
1137
1168
1. Save the file.
1169
+
1138
1170
1. Run this command:
1139
1171
1140
1172
```azurecli
@@ -1164,15 +1196,18 @@ If you want to update the code, model, or environment, update the configuration,
1164
1196
To understand how `begin_create_or_update` works:
1165
1197
1166
1198
1. Open the file *online/model-1/onlinescoring/score.py*.
1167
-
2. Change the last line of the `init()` function: After `logging.info("Init complete")`, add `logging.info("Updated successfully")`.
1168
-
3. Save the file.
1169
-
4. Run the method:
1199
+
1200
+
1. Change the last line of the `init()` function: After `logging.info("Init complete")`, add `logging.info("Updated successfully")`.
5. Because you modified the `init()` function, which runs when the endpoint is created or updated, the message `Updated successfully` will be in the logs. Retrieve the logs by running:
1210
+
1. Because you modified the `init()` function, which runs when the endpoint is created or updated, the message `Updated successfully` will be in the logs. Retrieve the logs by running:
1176
1211
1177
1212
```python
1178
1213
ml_client.online_deployments.get_logs(
@@ -1187,10 +1222,13 @@ The `begin_create_or_update` method also works with local deployments. Use the s
1187
1222
Currently, the studio allows you to make updates only to the instance count of a deployment. Use the following instructions to scale an individual deployment up or down by adjusting the number of instances:
1188
1223
1189
1224
1. Open the endpoint's **Details** page and find the card for the deployment you want to update.
1225
+
1190
1226
1. Select the edit icon (pencil icon) next to the deployment's name.
1227
+
1191
1228
1. Update the instance count associated with the deployment. You can choose between **Default** or **Target Utilization** for **Deployment scale type**.
1192
-
- If you select **Default**, you cal also specify a numerical value for the **Instance count**.
1229
+
- If you select **Default**, you can also specify a numerical value for the **Instance count**.
1193
1230
- If you select **Target Utilization**, you can specify values to use for parameters when autoscaling the deployment.
1231
+
1194
1232
1. Select **Update** to finish updating the instance counts for your deployment.
If you aren't going use the endpoint and deployment, you should delete them. By deleting the endpoint, you also delete all its underlying deployments.
1241
1279
1242
1280
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
1243
-
1. In the left pane, select the **Endpoints** page.
1281
+
1282
+
1. In the side pane, select the **Endpoints** page.
1283
+
1244
1284
1. Select an endpoint by checking the circle next to the model name.
1285
+
1245
1286
1. Select **Delete**.
1246
1287
1247
1288
Alternatively, you can delete a managed online endpoint directly by selecting the **Delete** icon in the [endpoint details page](#check-the-status-of-the-endpoint).
0 commit comments