Skip to content

Commit bb53e42

Browse files
committed
edit mirroring limitations
1 parent f41135c commit bb53e42

File tree

1 file changed

+8
-7
lines changed

1 file changed

+8
-7
lines changed

articles/machine-learning/how-to-safely-rollout-online-endpoints.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -566,18 +566,19 @@ Though `green` has 0% of traffic allocated, you can still invoke the endpoint an
566566
## Test the deployment with mirrored traffic (preview)
567567
[!INCLUDE [preview disclaimer](../../includes/machine-learning-preview-generic-disclaimer.md)]
568568

569-
Once you've tested your `green` deployment, you can 'mirror' (or copy) a percentage of the live traffic to it. Mirroring traffic (also called shadowing) doesn't change the results returned to clients. Requests still flow 100% to the `blue` deployment. The mirrored percentage of the traffic is copied and submitted to the `green` deployment so you can gather metrics and logging without impacting your clients. Mirroring is useful when you want to validate a new deployment without impacting clients. For example, you can use mirroring to check if latency is within acceptable bounds or to check that there are no HTTP errors. Testing the new deployment with traffic mirroring/shadowing is also known as [shadow testing](https://microsoft.github.io/code-with-engineering-playbook/automated-testing/shadow-testing/). The deployment receiving the mirrored traffic (in this case, the `green` deployment) can also be called the shadow deployment.
569+
Once you've tested your `green` deployment, you can *mirror* (or copy) a percentage of the live traffic to it. Traffic mirroring (also called shadowing) doesn't change the results returned to clients. Requests still flow 100% to the `blue` deployment. The mirrored percentage of the traffic is copied and submitted to the `green` deployment so that you can gather metrics and logging without impacting your clients. Mirroring is useful when you want to validate a new deployment without impacting clients. For example, you can use mirroring to check if latency is within acceptable bounds or to check that there are no HTTP errors. Testing the new deployment with traffic mirroring/shadowing is also known as [shadow testing](https://microsoft.github.io/code-with-engineering-playbook/automated-testing/shadow-testing/). The deployment receiving the mirrored traffic (in this case, the `green` deployment) can also be called the shadow deployment.
570570

571571
Mirroring has the following limitations:
572-
* Mirrored traffic is supported for the CLI (v2) (version 2.4.0 or above) and Python SDK (v2) (version 1.0.0 or above). If you update the endpoint using an older version of CLI/SDK or Studio UI, you'll lose the setting for mirrored traffic.
573-
* Mirrored traffic isn't currently supported for Kubernetes online endpoints.
572+
* Mirroring is supported for the CLI (v2) (version 2.4.0 or above) and Python SDK (v2) (version 1.0.0 or above). If you update an endpoint using an older version of CLI/SDK or Studio UI, you'll lose the mirrored traffic setting.
573+
* Mirroring isn't currently supported for Kubernetes online endpoints.
574574
* You can mirror traffic to only one deployment.
575575
* The maximum mirrored traffic you can configure is 50%. This limit is to reduce the effect on your [endpoint bandwidth quota](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints) (default 5 MBPS)—your endpoint bandwidth is throttled if you exceed the allocated quota. For information on monitoring bandwidth throttling, see [Monitor managed online endpoints](how-to-monitor-online-endpoints.md#metrics-at-endpoint-scope).
576576

577577
Also note the following behavior:
578-
* A deployment can only be set to live or mirrored traffic, not both.
579-
* You can send traffic directly to the mirror deployment by specifying the deployment set for mirror traffic.
580-
* You can send traffic directly to a live deployment by specifying the deployment set for live traffic, but in this case the traffic is not mirrored to the mirror deployment. Mirror traffic is routed from traffic sent to the endpoint without specifying the deployment.
578+
579+
* A deployment can be configured to receive only live traffic or mirrored traffic, not both.
580+
* You can send traffic directly to the shadow deployment (that is, a deployment that is set to receive mirrored traffic) by specifying the shadow deployment when you invoke the endpoint. Continuing with our example where the green deployment is the shadow deployment, you can invoke the endpoint with the specification that it sends back the response of the green deployment.
581+
* You can send traffic directly to a deployment that is already configured to receive live traffic by specifying the deployment when you invoke the endpoint. However, specifying a deployment to receive live traffic will prevent Azure Machine Learning from mirroring traffic to the shadow deployment. Azure Machine Learning routes mirrored traffic to the shadow deployment from traffic sent to the endpoint when you don't specify a deployment.
581582

582583
> [!TIP]
583584
> You can use `--deployment-name` option [for CLI v2](/cli/azure/ml/online-endpoint#az-ml-online-endpoint-invoke-optional-parameters), or `deployment_name` option [for SDK v2](/python/api/azure-ai-ml/azure.ai.ml.operations.onlineendpointoperations#azure-ai-ml-operations-onlineendpointoperations-invoke) to specify the deployment to be routed to.
@@ -647,7 +648,7 @@ The endpoint details page now shows mirrored traffic allocation of 10% to the `g
647648

648649
:::image type="content" source="media/how-to-safely-rollout-managed-endpoints/endpoint-details-showing-mirrored-traffic-allocation.png" alt-text="Endpoint details page showing mirrored traffic allocation in the deployment summary." lightbox="media/how-to-safely-rollout-managed-endpoints/endpoint-details-showing-mirrored-traffic-allocation.png":::
649650

650-
<!-- M.A.: Test mirrored traffic and confirm that the specific percentage of traffic was mirrored to the green deployment -->
651+
<!-- M.A.: Invoke the endpoint several times and confirm that the specific percentage of traffic was mirrored to the green deployment -->
651652

652653
After testing, you can disable mirroring:
653654

0 commit comments

Comments
 (0)