Skip to content

Commit c968fbf

Browse files
committed
add section for scaling deployment
1 parent 06757cc commit c968fbf

File tree

2 files changed

+14
-14
lines changed

2 files changed

+14
-14
lines changed

articles/machine-learning/how-to-safely-rollout-online-endpoints.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -217,7 +217,7 @@ To create an online endpoint:
217217

218218
### Create the 'blue' deployment
219219

220-
A deployment is a set of resources required for hosting the model that does the actual inferencing. In this article, you'll use the *endpoints/online/managed/sample/blue-deployment.yml* file to configure the key aspects of the deployment:<!-- [link to "define the deployment" section in Deploy article] -->. The following snippet shows the contents of the file:
220+
A deployment is a set of resources required for hosting the model that does the actual inferencing. In this article, you'll use the *endpoints/online/managed/sample/blue-deployment.yml* file to configure the key aspects of the deployment<!-- [link to "define the deployment" section in Deploy article] -->. The following snippet shows the contents of the file:
221221

222222
:::code language="yaml" source="~/azureml-examples-main/cli/endpoints/online/managed/sample/blue-deployment.yml":::
223223

@@ -311,7 +311,7 @@ For information on creating an environment in the studio, see [Create an environ
311311

312312
### Create a managed online endpoint and the 'blue' deployment
313313

314-
Use the Azure machine learning studio to create a managed online endpoint directly in your browser. When you create a managed online endpoint in the studio, you must define an initial deployment. You can't create an empty managed online endpoint.
314+
Use the Azure Machine Learning studio to create a managed online endpoint directly in your browser. When you create a managed online endpoint in the studio, you must define an initial deployment. You can't create an empty managed online endpoint.
315315

316316
One way to create a managed online endpoint in the studio is from the **Models** page. This method also provides an easy way to add a model to an existing managed online deployment. To deploy the model named `model-1` that you registered previously in the [Register the model](#register-the-model) section:
317317

@@ -450,7 +450,14 @@ Using the `MLClient` created earlier, we'll get a handle to the deployment. The
450450

451451
# [Studio](#tab/azure-studio)
452452

453-
M.A: add details here from "Update deployment instance count" section of "online endpoints in studio"
453+
Use the following instructions to scale the deployment up or down by adjusting the number of instances:
454+
455+
1. In the endpoint Details page. Find the card for the blue deployment.
456+
1. Select the **edit icon** in the header of the blue deployment's card.
457+
1. Change the instance count to 2.
458+
1. Select **Update**.
459+
460+
:::image type="content" source="media/how-to-safely-rollout-managed-endpoints/scale-blue-deployment.png" alt-text="A screenshot showing how to adjust the number of instances used by the blue deployment." lightbox="media/how-to-safely-rollout-managed-endpoints/scale-blue-deployment.png":::
454461

455462
---
456463

@@ -462,7 +469,7 @@ Create a new deployment named `green`:
462469

463470
:::code language="azurecli" source="~/azureml-examples-main/cli/deploy-safe-rollout-online-endpoints.sh" ID="create_green" :::
464471

465-
Since we haven't explicitly allocated any traffic to `green`, it will have zero traffic allocated to it. You can verify that using the command:
472+
Since we haven't explicitly allocated any traffic to `green`, it has zero traffic allocated to it. You can verify that using the command:
466473

467474
:::code language="azurecli" source="~/azureml-examples-main/cli/deploy-safe-rollout-online-endpoints.sh" ID="get_traffic" :::
468475

@@ -534,19 +541,12 @@ Though `green` has 0% of traffic allocated, you can still invoke the endpoint an
534541
1. Paste the sample input in the test box.
535542
1. Select **Test**.
536543

537-
538-
<!--
539-
> [!NOTE]
540-
> You can adjust the traffic balance between deployments in an endpoint when adding a new deployment.
541-
>
542-
> :::image type="content" source="media/how-to-create-managed-online-endpoint-studio/adjust-deployment-traffic.png" lightbox="media/how-to-create-managed-online-endpoint-studio/adjust-deployment-traffic.png" alt-text="A screenshot of how to use sliders to control traffic distribution across multiple deployments."::: -->
543-
544544
---
545545

546546
## Test the deployment with mirrored traffic (preview)
547547
[!INCLUDE [preview disclaimer](../../includes/machine-learning-preview-generic-disclaimer.md)]
548548

549-
Once you've tested your `green` deployment, you can 'mirror' (or copy) a percentage of the live traffic to it. Mirroring traffic (also called shadowing) doesn't change the results returned to clients. Requests still flow 100% to the `blue` deployment. The mirrored percentage of the traffic is copied and submitted to the `green` deployment so you can gather metrics and logging without impacting your clients. Mirroring is useful when you want to validate a new deployment without impacting clients; for example, to check if latency is within acceptable bounds and that there are no HTTP errors. Testing the new deployment with traffic mirroring/shadowing is also known as [shadow testing](https://microsoft.github.io/code-with-engineering-playbook/automated-testing/shadow-testing/). The deployment receiving the mirrored traffic (in this case, the `green` deployment) can also be called the shadow deployment.
549+
Once you've tested your `green` deployment, you can 'mirror' (or copy) a percentage of the live traffic to it. Mirroring traffic (also called shadowing) doesn't change the results returned to clients. Requests still flow 100% to the `blue` deployment. The mirrored percentage of the traffic is copied and submitted to the `green` deployment so you can gather metrics and logging without impacting your clients. Mirroring is useful when you want to validate a new deployment without impacting clients. For example, you can use mirroring to check if latency is within acceptable bounds or to check that there are no HTTP errors. Testing the new deployment with traffic mirroring/shadowing is also known as [shadow testing](https://microsoft.github.io/code-with-engineering-playbook/automated-testing/shadow-testing/). The deployment receiving the mirrored traffic (in this case, the `green` deployment) can also be called the shadow deployment.
550550

551551
> [!WARNING]
552552
> Mirroring traffic uses your [endpoint bandwidth quota](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints) (default 5 MBPS). Your endpoint bandwidth will be throttled if you exceed the allocated quota. For information on monitoring bandwidth throttling, see [Monitor managed online endpoints](how-to-monitor-online-endpoints.md#metrics-at-endpoint-scope).
@@ -586,7 +586,7 @@ The studio doesn't support mirrored traffic. See the Azure CLI or Python tabs fo
586586
Mirroring has the following limitations:
587587
* You can only mirror traffic to one deployment.
588588
* Mirror traffic isn't currently supported for Kubernetes online endpoints.
589-
* The maximum mirrored traffic you can configure is 50%. This limit is to reduce the impact on your endpoint bandwidth quota.
589+
* The maximum mirrored traffic you can configure is 50%. This limit is to reduce the effect on your endpoint bandwidth quota.
590590

591591
Also note the following behavior:
592592
* A deployment can only be set to live or mirror traffic, not both.
@@ -685,7 +685,7 @@ Once you're fully satisfied with your `green` deployment, switch all traffic to
685685

686686
## Remove the old deployment
687687

688-
Use the following steps to delete an individual deployment from a managed online endpoint. This does affect the other deployments in the managed online endpoint:
688+
Use the following steps to delete an individual deployment from a managed online endpoint. Deleting an individual deployment does affect the other deployments in the managed online endpoint:
689689

690690
# [Azure CLI](#tab/azure-cli)
691691

16.5 KB
Loading

0 commit comments

Comments
 (0)