Skip to content

Commit 362d264

Browse files
authored
Add mirror traffic section in SDKv2 doc
Add mirror traffic section in SDKv2 doc, reflect the contents in azureml-examples Also add some images exists in CLIv2 version of this page.
1 parent e316394 commit 362d264

File tree

1 file changed

+61
-20
lines changed

1 file changed

+61
-20
lines changed

articles/machine-learning/how-to-safely-rollout-managed-endpoints-sdk-v2.md

Lines changed: 61 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -256,7 +256,7 @@ green_deployment = ManagedOnlineDeployment(
256256
ml_client.begin_create_or_update(green_deployment)
257257
```
258258

259-
## Test the 'green' deployment
259+
### Test the new deployment
260260

261261
Though green has 0% of traffic allocated, you can still invoke the endpoint and deployment with [json](https://github.com/Azure/azureml-examples/blob/main/sdk/endpoints/online/model-2/sample-request.json) file.
262262

@@ -268,38 +268,79 @@ ml_client.online_endpoints.invoke(
268268
)
269269
```
270270

271-
1. Test the new deployment with a small percentage of live traffic:
271+
## Test the deployment with mirrored traffic (preview)
272272

273-
Once you've tested your green deployment, allocate a small percentage of traffic to it:
273+
[!INCLUDE [preview disclaimer](../../includes/machine-learning-preview-generic-disclaimer.md)]
274274

275-
```python
276-
endpoint.traffic = {"blue": 90, "green": 10}
277-
ml_client.begin_create_or_update(endpoint)
278-
```
275+
Once you've tested your `green` deployment, you can copy (or 'mirror') a percentage of the live traffic to it. Mirroring traffic doesn't change results returned to clients. Requests still flow 100% to the blue deployment. The mirrored percentage of the traffic is copied and submitted to the `green` deployment so you can gather metrics and logging without impacting your clients. Mirroring is useful when you want to validate a new deployment without impacting clients. For example, to check if latency is within acceptable bounds and that there are no HTTP errors.
279276

280-
Now, your green deployment will receive 10% of requests.
277+
> [!WARNING]
278+
> Mirroring traffic uses your [endpoint bandwidth quota](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints) (default 5 MBPS). Your endpoint bandwidth will be throttled if you exceed the allocated quota. For information on monitoring bandwidth throttling, see [Monitor managed online endpoints](how-to-monitor-online-endpoints.md#metrics-at-endpoint-scope).
281279

282-
1. Send all traffic to your new deployment:
280+
The following command mirrors 10% of the traffic to the `green` deployment:
283281

284-
Once you're satisfied that your green deployment is fully satisfactory, switch all traffic to it.
282+
```python
283+
endpoint.mirror_traffic = {"green": 10}
284+
ml_client.begin_create_or_update(endpoint)
285+
```
285286

286-
```python
287-
endpoint.traffic = {"blue": 0, "green": 100}
288-
ml_client.begin_create_or_update(endpoint)
289-
```
287+
> [!IMPORTANT]
288+
> Mirroring has the following limitations:
289+
> * You can only mirror traffic to one deployment.
290+
> * A deployment can only be set to live or mirror traffic, not both.
291+
> * Mirrored traffic is not currently supported with K8s.
292+
> * The maximum mirrored traffic you can configure is 50%. This limit is to reduce the impact on your endpoint bandwidth quota.
290293

291-
1. Remove the old deployment:
294+
:::image type="content" source="./media/how-to-safely-rollout-managed-endpoints/endpoint-concept-mirror.png" alt-text="Diagram showing 10% traffic mirrored to one deployment.":::
292295

293-
```python
294-
ml_client.online_deployments.delete(name="blue", endpoint_name=online_endpoint_name)
295-
```
296+
After testing, you can set the mirror traffic to zero to disable mirroring:
297+
298+
```python
299+
endpoint.mirror_traffic = {"green": 0}
300+
ml_client.begin_create_or_update(endpoint)
301+
```
302+
303+
## Test the new deployment with a small percentage of live traffic:
304+
305+
Once you've tested your green deployment, allocate a small percentage of traffic to it:
306+
307+
```python
308+
endpoint.traffic = {"blue": 90, "green": 10}
309+
ml_client.begin_create_or_update(endpoint)
310+
```
311+
312+
Now, your green deployment will receive 10% of requests.
313+
314+
:::image type="content" source="./media/how-to-safely-rollout-managed-endpoints/endpoint-concept.png" alt-text="Diagram showing traffic split between deployments.":::
315+
316+
## Send all traffic to your new deployment:
317+
318+
Once you're satisfied that your green deployment is fully satisfactory, switch all traffic to it.
319+
320+
```python
321+
endpoint.traffic = {"blue": 0, "green": 100}
322+
ml_client.begin_create_or_update(endpoint)
323+
```
324+
325+
## Remove the old deployment:
326+
327+
```python
328+
ml_client.online_deployments.delete(name="blue", endpoint_name=online_endpoint_name)
329+
```
296330

297331
## Delete endpoint
298332

333+
If you aren't going use the deployment, you should delete it with:
334+
299335
```python
300336
ml_client.online_endpoints.begin_delete(name=online_endpoint_name)
301337
```
302338

303339
## Next steps
304-
305-
* Explore online endpoint samples - [https://github.com/Azure/azureml-examples/tree/main/sdk/endpoints](https://github.com/Azure/azureml-examples/tree/main/sdk/endpoints)
340+
- [Explore online endpoint samples](https://github.com/Azure/azureml-examples/tree/main/sdk/endpoints)
341+
- [Access Azure resources with a online endpoint and managed identity](how-to-access-resources-from-endpoints-managed-identities.md)
342+
- [Monitor managed online endpoints](how-to-monitor-online-endpoints.md)
343+
- [Manage and increase quotas for resources with Azure Machine Learning](how-to-manage-quotas.md#azure-machine-learning-managed-online-endpoints)
344+
- [View costs for an Azure Machine Learning managed online endpoint](how-to-view-online-endpoints-costs.md)
345+
- [Managed online endpoints SKU list](reference-managed-online-endpoints-vm-sku-list.md)
346+
- [Troubleshooting online endpoints de

0 commit comments

Comments
 (0)