Skip to content

Commit 4e779a1

Browse files
committed
Minor edits
1 parent 5006261 commit 4e779a1

File tree

1 file changed

+58
-17
lines changed

1 file changed

+58
-17
lines changed

articles/machine-learning/how-to-deploy-online-endpoints.md

Lines changed: 58 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ Before following the steps in this article, make sure you have the following pre
5353

5454
* An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://azure.microsoft.com/free/).
5555

56-
* An Azure Machine Learning workspace and a compute instance. If you don't have these resources and want to create them, see [Create resources you need to get started](quickstart-create-resources.md).
56+
* An Azure Machine Learning workspace and a compute instance. If you don't have these resources, see [Create resources you need to get started](quickstart-create-resources.md).
5757

5858
* Azure role-based access control (Azure RBAC) is used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the _owner_ or _contributor_ role for the Azure Machine Learning workspace, or a custom role allowing `Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*`. For more information, see [Manage access to an Azure Machine Learning workspace](how-to-assign-roles.md).
5959

@@ -68,7 +68,7 @@ Before following the steps in this article, make sure you have the following pre
6868

6969
---
7070

71-
* Ensure that you have enough virtual machine (VM) quota allocated for deployment. Azure Machine Learning reserves 20% of your compute resources for performing upgrades on some VM SKUs. For example, if you request 10 instances in a deployment, you must have a quota for 12 for each number of cores for the VM SKU. Failure to account for the extra compute resources results in an error. There are some VM SKUs that are exempt from the extra quota reservation. For more information on quota allocation, see [virtual machine quota allocation for deployment](how-to-manage-quotas.md#virtual-machine-quota-allocation-for-deployment).
71+
* Ensure that you have enough virtual machine (VM) quota allocated for deployment. Azure Machine Learning reserves 20% of your compute resources for performing upgrades on some VM SKUs. For example, if you request 10 instances in a deployment, you must have a quota of 12 for each number of cores for the VM SKU. Failure to account for the extra compute resources results in an error. There are some VM SKUs that are exempt from the extra quota reservation. For more information on quota allocation, see [virtual machine quota allocation for deployment](how-to-manage-quotas.md#virtual-machine-quota-allocation-for-deployment).
7272

7373
* Alternatively, you could use quota from Azure Machine Learning's shared quota pool for a limited time. [!INCLUDE [machine-learning-shared-quota](includes/machine-learning-shared-quota.md)]
7474

@@ -87,7 +87,7 @@ If you haven't already set the defaults for the Azure CLI, save your default set
8787

8888
### Clone the examples repository
8989

90-
To follow along with this article, first clone the [azureml-examples repository](https://github.com/azure/azureml-examples), then change into the repository's *azureml-examples/cli*` directory:
90+
To follow along with this article, first clone the [azureml-examples repository](https://github.com/azure/azureml-examples), then change into the repository's *azureml-examples/cli* directory:
9191

9292
```azurecli
9393
git clone --depth 1 https://github.com/Azure/azureml-examples
@@ -179,7 +179,9 @@ cd azureml-examples/cli/endpoints/online/model-1
179179
If you cloned the examples repo, your local machine already has copies of the files for this example, and you can skip to the next section. If you didn't clone the repo, you can download it to your local machine.
180180

181181
1. Go to [https://github.com/Azure/azureml-examples/](https://github.com/Azure/azureml-examples/).
182+
182183
1. Go to the **<> Code** button on the page, and then select **Download ZIP** from the **Local** tab.
184+
183185
1. Locate the folder */cli/endpoints/online/model-1/model* and the file */cli/endpoints/online/model-1/onlinescoring/score.py*.
184186

185187
# [ARM template](#tab/arm)
@@ -258,15 +260,15 @@ First define the name of the online endpoint, then configure the endpoint.
258260
Replace `<YOUR_ENDPOINT_NAME>` with a name that's unique in the Azure region, or use the example method to define a random name. Be sure to delete the method that you don't use. For more information on the naming rules, see [endpoint limits](how-to-manage-quotas.md#azure-machine-learning-online-endpoints-and-batch-endpoints).
259261

260262
```python
261-
# Method 1: Define an endpoint name
263+
# method 1: define an endpoint name
262264
endpoint_name = "<YOUR_ENDPOINT_NAME>"
263265

264-
# Method 2: Example way to define a random name
266+
# method 2: example way to define a random name
265267
import datetime
266268

267269
endpoint_name = "endpt-" + datetime.datetime.now().strftime("%m%d%H%M%f")
268270

269-
# Create an online endpoint
271+
# create an online endpoint
270272
endpoint = ManagedOnlineEndpoint(
271273
name = endpoint_name,
272274
description="this is a sample endpoint",
@@ -692,31 +694,43 @@ A model registration is a logical entity in the workspace that can contain a sin
692694
To register the example model, follow these steps:
693695

694696
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
695-
1. In the left pane, select the **Models** page.
697+
698+
1. In the side pane, select the **Models** page.
699+
696700
1. Select **Register**, and then choose **From local files**.
701+
697702
1. Select __Unspecified type__ for the __Model type__.
703+
698704
1. Select __Browse__, and choose __Browse folder__.
699705

700706
:::image type="content" source="media/how-to-deploy-online-endpoints/register-model-folder.png" alt-text="A screenshot of the browse folder option." lightbox="media/how-to-deploy-online-endpoints/register-model-folder.png":::
701707

702708
1. Select the `\azureml-examples\cli\endpoints\online\model-1\model` folder from the local copy of the repo you cloned or downloaded earlier. When prompted, select __Upload__ and wait for the upload to complete.
709+
703710
1. Select __Next__ after the folder upload is completed.
711+
704712
1. Enter a friendly __Name__ for the model. The steps in this article assume the model is named `model-1`.
713+
705714
1. Select __Next__, and then __Register__ to complete registration.
706715

707716
For more information on working with registered models, see [Work with registered models](how-to-manage-models.md).
708717

709718
### Create and register the environment
710719

711720
1. In the side navigation bar, select the **Environments** page.
721+
712722
1. Select the **Custom environments** tab, then choose **Create**.
723+
713724
1. On the **Settings** page, provide a name, such as *my-env* for the environment.
725+
714726
1. For **Select environment source**, choose **Use existing docker image with optional conda source**.
715727

716728
:::image type="content" source="media/how-to-deploy-online-endpoints/create-environment.png" alt-text="A screenshot showing how to create a custom environment." lightbox="media/how-to-deploy-online-endpoints/create-environment.png":::
717729

718730
1. Select **Next** to go to the **Customize** page.
731+
719732
1. Copy the contents of the *\azureml-examples\cli\endpoints\online\model-1\environment\conda.yaml* file from the repo you cloned or downloaded earlier.
733+
720734
1. Paste the contents into the text box.
721735

722736
:::image type="content" source="media/how-to-deploy-online-endpoints/customize-environment-with-conda-file.png" alt-text="A screenshot showing how to customize the environment, using a conda file." lightbox="media/how-to-deploy-online-endpoints/customize-environment-with-conda-file.png":::
@@ -754,7 +768,7 @@ Your deployment configuration uses the registered model that you wish to deploy
754768

755769
# [Azure CLI](#tab/cli)
756770

757-
Use the registered assets (model and environment) in your deployment definition. The following snippet shows the `endpoints/online/managed/sample/blue-deployment-with-registered-assets.yml` file, with all the required inputs to configure a deployment:
771+
Use the registered assets (model and environment) in your deployment definition. The following snippet shows the *endpoints/online/managed/sample/blue-deployment-with-registered-assets.yml* file, with all the required inputs to configure a deployment:
758772

759773
__blue-deployment-with-registered-assets.yml__
760774

@@ -889,20 +903,27 @@ Use the studio to create a managed online endpoint directly in your browser. Whe
889903
One way to create a managed online endpoint in the studio is from the **Models** page. This method also provides an easy way to add a model to an existing managed online deployment. To deploy the model named `model-1` that you registered previously in the [Register your model and environment](#register-your-model-and-environment) section:
890904

891905
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
906+
892907
1. In the side navigation bar, select the **Models** page.
908+
893909
1. Select the model named `model-1` by checking the circle next to its name.
910+
894911
1. Select **Deploy** > **Real-time endpoint**.
895912

896913
:::image type="content" source="media/how-to-deploy-online-endpoints/deploy-from-models-page.png" lightbox="media/how-to-deploy-online-endpoints/deploy-from-models-page.png" alt-text="A screenshot of creating a managed online endpoint from the Models UI.":::
897-
914+
898915
This action opens up a window where you can specify details about your endpoint.
899916

900917
:::image type="content" source="media/how-to-deploy-online-endpoints/online-endpoint-wizard.png" lightbox="media/how-to-deploy-online-endpoints/online-endpoint-wizard.png" alt-text="A screenshot of a managed online endpoint create wizard.":::
901918

902919
1. Enter an __Endpoint name__ that's unique in the Azure region. For more information on the naming rules, see [endpoint limits](how-to-manage-quotas.md#azure-machine-learning-online-endpoints-and-batch-endpoints).
920+
903921
1. Keep the default selection: __Managed__ for the compute type.
922+
904923
1. Keep the default selection: __key-based authentication__ for the authentication type. For more information on authenticating, see [Authenticate clients for online endpoints](how-to-authenticate-online-endpoint.md).
924+
905925
1. Select __Next__, until you get to the **Deployment** page. Here, toggle __Application Insights diagnostics__ to *Enabled* to allow you to view graphs of your endpoint's activities in the studio later and analyze metrics and logs using Application Insights.
926+
906927
1. Select __Next__ to go to the **Code + environment** page. Here, select the following options:
907928

908929
* __Select a scoring script for inferencing__: Browse and select the *\azureml-examples\cli\endpoints\online\model-1\onlinescoring\score.py* file from the repo you cloned or downloaded earlier.
@@ -911,12 +932,15 @@ One way to create a managed online endpoint in the studio is from the **Models**
911932
:::image type="content" source="media/how-to-deploy-online-endpoints/deploy-with-custom-environment.png" lightbox="media/how-to-deploy-online-endpoints/deploy-with-custom-environment.png" alt-text="A screenshot showing selection of a custom environment for deployment.":::
912933

913934
1. Select __Next__, accepting defaults, until you're prompted to create the deployment.
935+
914936
1. Review your deployment settings and select the __Create__ button.
915937

916938
Alternatively, you can create a managed online endpoint from the **Endpoints** page in the studio.
917939

918940
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
919-
1. In the left pane, select the **Endpoints** page.
941+
942+
1. In the side pane, select the **Endpoints** page.
943+
920944
1. Select **+ Create**.
921945

922946
:::image type="content" source="media/how-to-deploy-online-endpoints/endpoint-create-managed-online-endpoint.png" lightbox="media/how-to-deploy-online-endpoints/endpoint-create-managed-online-endpoint.png" alt-text="A screenshot for creating managed online endpoint from the Endpoints tab.":::
@@ -937,7 +961,7 @@ This action opens up a window for you to select your model and specify details a
937961

938962
To debug errors in your deployment, see [Troubleshooting online endpoint deployments](./how-to-troubleshoot-online-endpoints.md).
939963

940-
### Check the status of the endpoint
964+
### Check the status of the online endpoint
941965

942966
# [Azure CLI](#tab/cli)
943967

@@ -982,7 +1006,9 @@ To debug errors in your deployment, see [Troubleshooting online endpoint deploym
9821006
You can view all your managed online endpoints in the **Endpoints** page. Go to the endpoint's **Details** page to find critical information including the endpoint URI, status, testing tools, activity monitors, deployment logs, and sample consumption code:
9831007

9841008
1. In the side navigation bar, select **Endpoints**. Here, you can see a list of all the endpoints in the workspace.
1009+
9851010
1. (Optional) Create a **Filter** on **Compute type** to show only **Managed** compute types.
1011+
9861012
1. Select an endpoint name to view the endpoint's __Details__ page.
9871013

9881014
:::image type="content" source="media/how-to-deploy-online-endpoints/managed-endpoint-details-page.png" lightbox="media/how-to-deploy-online-endpoints/managed-endpoint-details-page.png" alt-text="Screenshot of managed endpoint details view.":::
@@ -1102,8 +1128,11 @@ Using the `MLClient` created earlier, get a handle to the endpoint. The endpoint
11021128
Use the **Test** tab in the endpoint's details page to test your managed online deployment. Enter sample input and view the results.
11031129

11041130
1. Select the **Test** tab in the endpoint's detail page.
1131+
11051132
1. Use the dropdown to select the deployment you want to test.
1133+
11061134
1. Enter the [sample input](https://github.com/Azure/azureml-examples/blob/main/sdk/python/endpoints/online/model-1/sample-request.json).
1135+
11071136
1. Select **Test**.
11081137

11091138
:::image type="content" source="media/how-to-deploy-online-endpoints/test-deployment.png" lightbox="media/how-to-deploy-online-endpoints/test-deployment.png" alt-text="A screenshot of testing a deployment by providing sample data, directly in your browser.":::
@@ -1133,8 +1162,11 @@ If you want to update the code, model, or environment, update the YAML file, and
11331162
To understand how `update` works:
11341163

11351164
1. Open the file *online/model-1/onlinescoring/score.py*.
1165+
11361166
1. Change the last line of the `init()` function: After `logging.info("Init complete")`, add `logging.info("Updated successfully")`.
1167+
11371168
1. Save the file.
1169+
11381170
1. Run this command:
11391171

11401172
```azurecli
@@ -1164,15 +1196,18 @@ If you want to update the code, model, or environment, update the configuration,
11641196
To understand how `begin_create_or_update` works:
11651197

11661198
1. Open the file *online/model-1/onlinescoring/score.py*.
1167-
2. Change the last line of the `init()` function: After `logging.info("Init complete")`, add `logging.info("Updated successfully")`.
1168-
3. Save the file.
1169-
4. Run the method:
1199+
1200+
1. Change the last line of the `init()` function: After `logging.info("Init complete")`, add `logging.info("Updated successfully")`.
1201+
1202+
1. Save the file.
1203+
1204+
1. Run the method:
11701205

11711206
```python
11721207
ml_client.online_deployments.begin_create_or_update(blue_deployment_with_registered_assets)
11731208
```
11741209

1175-
5. Because you modified the `init()` function, which runs when the endpoint is created or updated, the message `Updated successfully` will be in the logs. Retrieve the logs by running:
1210+
1. Because you modified the `init()` function, which runs when the endpoint is created or updated, the message `Updated successfully` will be in the logs. Retrieve the logs by running:
11761211

11771212
```python
11781213
ml_client.online_deployments.get_logs(
@@ -1187,10 +1222,13 @@ The `begin_create_or_update` method also works with local deployments. Use the s
11871222
Currently, the studio allows you to make updates only to the instance count of a deployment. Use the following instructions to scale an individual deployment up or down by adjusting the number of instances:
11881223

11891224
1. Open the endpoint's **Details** page and find the card for the deployment you want to update.
1225+
11901226
1. Select the edit icon (pencil icon) next to the deployment's name.
1227+
11911228
1. Update the instance count associated with the deployment. You can choose between **Default** or **Target Utilization** for **Deployment scale type**.
1192-
- If you select **Default**, you cal also specify a numerical value for the **Instance count**.
1229+
- If you select **Default**, you can also specify a numerical value for the **Instance count**.
11931230
- If you select **Target Utilization**, you can specify values to use for parameters when autoscaling the deployment.
1231+
11941232
1. Select **Update** to finish updating the instance counts for your deployment.
11951233

11961234
# [ARM template](#tab/arm)
@@ -1240,8 +1278,11 @@ ml_client.online_endpoints.begin_delete(name=endpoint_name)
12401278
If you aren't going use the endpoint and deployment, you should delete them. By deleting the endpoint, you also delete all its underlying deployments.
12411279

12421280
1. Go to the [Azure Machine Learning studio](https://ml.azure.com).
1243-
1. In the left pane, select the **Endpoints** page.
1281+
1282+
1. In the side pane, select the **Endpoints** page.
1283+
12441284
1. Select an endpoint by checking the circle next to the model name.
1285+
12451286
1. Select **Delete**.
12461287

12471288
Alternatively, you can delete a managed online endpoint directly by selecting the **Delete** icon in the [endpoint details page](#check-the-status-of-the-endpoint).

0 commit comments

Comments
 (0)