Merge pull request #114352 from MicrosoftDocs/j-martens-patch-27-1

denrea · web-flow · commit 5b4930c5665d · 2020-05-07T15:42:21.000-07:00
Update how-to-use-event-grid.md
diff --git a/articles/machine-learning/how-to-use-event-grid.md b/articles/machine-learning/how-to-use-event-grid.md
@@ -1,7 +1,7 @@
 ---
-title: Create event driven machine learning workflows 
+title: Trigger events in ML workflows 
 titleSuffix: Azure Machine Learning
-description: Learn how to use event grid with Azure Machine Learning to enable event driven solutions.
+description: Learn how to trigger event-driven applications, processes, or CI/CD workflows based on Azure Machine Learning events to simplify your ML lifecycle.
 services: machine-learning
 ms.service: machine-learning
 ms.subservice: core
@@ -12,27 +12,101 @@ ms.reviewer: larryfr
 ms.date: 03/11/2020
 ---
 
+# Trigger event-driven applications, processes, or CI/CD workflows based on Azure Machine Learning events (Preview)
 
-# Create event driven machine learning workflows (Preview)
+In this article, you learn how to set up event-driven applications, processes, or CI/CD workflows based on Azure Machine Learning events, such as failure notification emails or ML pipeline runs, when certain conditions are detected by [Azure Event Grid](https://docs.microsoft.com/azure/event-grid/). 
 
-[Azure Event Grid](https://docs.microsoft.com/azure/event-grid/) supports Azure Machine Learning events. You can subscribe and consume events such as run status changed, run completion, model registration, model deployment, and data drift detection within a workspace.
-
-
-Use Event Grid to enable common scenarios such as:
+Azure Machine Learning manages the entire lifecycle of machine learning process, including model training, model deployment, and monitoring. You can use Event Grid to react to Azure Machine Learning events, such as the completion of training runs, the registration and deployment of models, and the detection of data drift, by using modern serverless architectures. You can then subscribe and consume events such as run status changed, run completion, model registration, model deployment, and data drift detection within a workspace.
 
+When to use Event Grid for event driven actions:
 * Send emails on run failure and run completion
 * Use an Azure function after a model is registered
 * Streaming events from Azure Machine Learning to various of endpoints
 * Trigger an ML pipeline when drift is detected
 
 > [!NOTE] 
 > Currently, runStatusChanged events only trigger when the run status is **failed**
->
 
 ## Prerequisites
-* Contributor or owner access to the Azure Machine Learning workspace you will create events for.
+To use Event Grid, you need contributor or owner access to the Azure Machine Learning workspace you will create events for.
+
+## The event model & types
+
+Azure Event Grid reads events from sources, such as Azure Machine Learning and other Azure services. These events are then sent to event handlers such as Azure Event Hubs, Azure Functions, Logic Apps, and others. The following diagram shows how Event Grid connects sources and handlers, but is not a comprehensive list of supported integrations.
+
+![Azure Event Grid functional model](./media/concept-event-grid-integration/azure-event-grid-functional-model.png)
+
+For more information on event sources and event handlers, see [What is Event Grid?](/azure/event-grid/overview).
+
+### Event types for Azure Machine Learning
+
+Azure Machine Learning provides events in the various points of machine learning lifecycle: 
+
+| Event type | Description |
+| ---------- | ----------- |
+| `Microsoft.MachineLearningServices.RunCompleted` | Raised when a machine learning experiment run is completed |
+| `Microsoft.MachineLearningServices.ModelRegistered` | Raised when a machine learning model is registered in the workspace |
+| `Microsoft.MachineLearningServices.ModelDeployed` | Raised when a deployment of inference service with one or more models is completed |
+| `Microsoft.MachineLearningServices.DatasetDriftDetected` | Raised when a data drift detection job for two datasets is completed |
+| `Microsoft.MachineLearningServices.RunStatusChanged` | Raised when a run status changed, currently only raised when a run status is 'failed' |
+
+### Filter & subscribe to events
+
+These events are published through Azure Event Grid. Using Azure portal, PowerShell or Azure CLI, customers can easily subscribe to events by [specifying one or more event types, and filtering conditions](/azure/event-grid/event-filtering). 
+
+When setting up your events, you can apply filters to only trigger on specific event data. In the example below, for run status changed events, you can filter by run types. The event only triggers when the criteria is met. Refer to the [Azure Machine Learning event grid schema](/azure/event-grid/event-schema-machine-learning) to learn about event data you can filter by. 
+
+Subscriptions for Azure Machine Learning events are protected by role-based access control (RBAC). Only [contributor or owner](how-to-assign-roles.md#default-roles) of a workspace can create, update, and delete event subscriptions.  Filters can be applied to event subscriptions either during the [creation](/cli/azure/eventgrid/event-subscription?view=azure-cli-latest) of the event subscription or at a later time. 
+
+
+1. Go to the Azure portal, select a new subscription or an existing one. 
+
+1. Select the filters tab and scroll down to Advanced filters. For the **Key** and **Value**, provide the property types you want to filter by. Here you can see the event will only trigger when the run type is a pipeline run or pipeline step run.  
+
+    :::image type="content" source="media/how-to-use-event-grid/select-event-filters.png" alt-text="filter events":::
+
 
-### Configure EventGrid using the Azure portal
++ **Filter by event type:** An event subscription can specify one or more Azure Machine Learning event types.
+
++ **Filter by event subject:** Azure Event Grid supports subject filters based on __begins with__ and __ends with__ matches, so that events with a matching subject are delivered to the subscriber. Different machine learning events have different subject format.
+
+  | Event type | Subject format | Sample subject |
+  | ---------- | ----------- | ----------- |
+  | `Microsoft.MachineLearningServices.RunCompleted` | `experiments/{ExperimentId}/runs/{RunId}` | `experiments/b1d7966c-f73a-4c68-b846-992ace89551f/runs/my_exp1_1554835758_38dbaa94` |
+  | `Microsoft.MachineLearningServices.ModelRegistered` | `models/{modelName}:{modelVersion}` | `models/sklearn_regression_model:3` |
+  | `Microsoft.MachineLearningServices.ModelDeployed` | `endpoints/{serviceId}` | `endpoints/my_sklearn_aks` |
+  | `Microsoft.MachineLearningServices.DatasetDriftDetected` | `datadrift/{data.DataDriftId}/run/{data.RunId}` | `datadrift/4e694bf5-712e-4e40-b06a-d2a2755212d4/run/my_driftrun1_1550564444_fbbcdc0f` |
+  | `Microsoft.MachineLearningServices.RunStatusChanged` | `experiments/{ExperimentId}/runs/{RunId}` | `experiments/b1d7966c-f73a-4c68-b846-992ace89551f/runs/my_exp1_1554835758_38dbaa94` | 
+
++ **Advanced filtering**: Azure Event Grid also supports advanced filtering based on published event schema. Azure Machine Learning event schema details can be found in [Azure Event Grid event schema for Azure Machine Learning](../event-grid/event-schema-machine-learning.md).  Some sample advanced filterings you can perform include:
+
+  For `Microsoft.MachineLearningServices.ModelRegistered` event, to filter model's tag value:
+
+  ```
+  --advanced-filter data.ModelTags.key1 StringIn ('value1')
+  ```
+
+  To learn more about how to apply filters, see [Filter events for Event Grid](https://docs.microsoft.com/azure/event-grid/how-to-filter-events).
+
+## Consume Machine Learning events
+
+Applications that handle Machine Learning events should follow a few recommended practices:
+
+> [!div class="checklist"]
+> * As multiple subscriptions can be configured to route events to the same event handler, it is important not to assume events are from a particular source, but to check the topic of the message to ensure that it comes from the machine learning workspace you are expecting.
+> * Similarly, check that the eventType is one you are prepared to process, and do not assume that all events you receive will be the types you expect.
+> * As messages can arrive out of order and after some delay, use the etag fields to understand if your information about objects is still up-to-date.  Also, use the sequencer fields to understand the order of events on any particular object.
+> * Ignore fields you don't understand. This practice will help keep you resilient to new features that might be added in the future.
+> * Failed or cancelled Azure Machine Learning operations will not trigger an event. For example, if a model deployment fails Microsoft.MachineLearningServices.ModelDeployed won't be triggered. Consider such failure mode when design your applications. You can always use Azure Machine Learning SDK, CLI or portal to check the status of an operation and understand the detailed failure reasons.
+
+Azure Event Grid allows customers to build de-coupled message handlers, which can be triggered by Azure Machine Learning events. Some notable examples of message handlers are:
+* Azure Functions
+* Azure Logic Apps
+* Azure Event Hubs
+* Azure Data Factory Pipeline
+* Generic webhooks, which may be hosted on the Azure platform or elsewhere
+
+## Set up in Azure portal
 
 1. Open the [Azure portal](https://portal.azure.com) and go to your Azure Machine Learning workspace.
 
@@ -51,7 +125,7 @@ Use Event Grid to enable common scenarios such as:
 Once you have confirmed your selection, click __Create__. After configuration, these events will be pushed to your endpoint.
 
 
-### Configure EventGrid using the CLI
+### Set up with the CLI
 
 You can either install the latest [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli?view=azure-cli-latest), or use the Azure Cloud Shell that is provided as part of your Azure subscription.
 
@@ -76,17 +150,9 @@ az eventgrid event-subscription create \
   --subject-begins-with "models/mymodelname"
 ```
 
-## Filter Events
+## Examples
 
-When setting up your events, you can apply filters to only trigger on specific event data. In the example below, for run status changed events, you can filter by run types. The event only triggers when the criteria is met. Refer to the [Azure Machine Learning event grid schema](/azure/event-grid/event-schema-machine-learning) to learn about event data you can filter by. 
-
-1. Go to the Azure portal, select a new subscription or an existing one. 
-
-1. Select the filters tab and scroll down to Advanced filters. For the **Key** and **Value**, provide the property types you want to filter by. Here you can see the event will only trigger when the run type is a pipeline run or pipeline step run.  
-
-    :::image type="content" source="media/how-to-use-event-grid/select-event-filters.png" alt-text="filter events":::
-
-## Sample: Send email alerts
+### Example: Send email alerts
 
 Use [Azure Logic Apps](https://docs.microsoft.com/azure/logic-apps/) to configure emails for all your events. Customize with conditions and specify recipients to enable collaboration and awareness across teams working together.
 
@@ -119,7 +185,7 @@ Use [Azure Logic Apps](https://docs.microsoft.com/azure/logic-apps/) to configur
     ![confirm-logic-app-create](./media/how-to-use-event-grid/confirm-logic-app-create.png)
 
 
-## Sample: Trigger retraining when data drift occurs
+### Example: Data drift triggers retraining
 
 Models go stale over time, and not remain useful in the context it is running in. One way to tell if it's time to retrain the model is detecting data drift. 
 
@@ -166,12 +232,17 @@ Now the data factory pipeline is triggered when drift occurs. View details on yo
 
 ![view-in-workspace](./media/how-to-use-event-grid/view-in-workspace.png)
 
-## Sample: Deploy a model based on tags
+### Example: Deploy a model based on tags
 
 An Azure Machine Learning model object contains parameters you can pivot deployments on such as model name, version, tag, and property. The model registration event can trigger an endpoint and you can use an Azure Function to deploy a model based on the value of those parameters.
 
 For an example, see the [https://github.com/Azure-Samples/MachineLearningSamples-NoCodeDeploymentTriggeredByEventGrid](https://github.com/Azure-Samples/MachineLearningSamples-NoCodeDeploymentTriggeredByEventGrid) repository and follow the steps in the **readme** file.
 
 ## Next steps
 
-* To learn more about available events, see the [Azure Machine Learning event schema](/azure/event-grid/event-schema-machine-learning)
+Learn more about Event Grid and give Azure Machine Learning events a try:
+
+- [About Event Grid](../event-grid/overview.md)
+
+- [Event schema for Azure Machine Learning](../event-grid/event-schema-machine-learning.md)
+
