Skip to content

Commit 9b6aabb

Browse files
Ja-DunnJa-Dunn
authored
Merge pull request #94486 from Blackmist/event-grid
new files for event grid + azure machine learning integration
2 parents 2b73236 + ef960cd commit 9b6aabb

33 files changed

+499
-0
lines changed

‎articles/event-grid/event-schema-machine-learning.md

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
---
2+
title: Azure Event Grid Machine Learning event schema
3+
description: Describes the properties that are provided for Machine Learning Workspace events with Azure Event Grid
4+
services: event-grid
5+
author: jenns
6+
7+
ms.service: event-grid
8+
ms.topic: reference
9+
ms.date: 10/18/2019
10+
ms.author: jenns
11+
---
12+
13+
# Azure Event Grid event schema for Azure Machine Learning
14+
15+
This article provides the properties and schema for machine learning workspace events. For an introduction to event schemas, see [Azure Event Grid event schema](event-schema.md).
16+
17+
For a list of sample scripts and tutorials, see [AzureML event source](event-sources.md#azure-machine-learning).
18+
19+
## Available event types
20+
21+
Azure Machine Learning emits the following event types:
22+
23+
| Event type | Description |
24+
| ---------- | ----------- |
25+
| Microsoft.MachineLearningServices.ModelRegistered | Raised when a new Model or Model version has been successfully registered. |
26+
| Microsoft.MachineLearningServices.ModelDeployed | Raised when Model(s) have been successfully deployed to an Endpoint. |
27+
| Microsoft.MachineLearningServices.RunCompleted | Raised when a Run has been successfully completed. |
28+
| Microsoft.MachineLearningServices.DatasetDriftDetected | Raised when a Dataset drift monitor detects drift. |
29+
30+
## The contents of an event response
31+
32+
When an event is triggered, the Event Grid service sends data about that event to subscribing endpoint.
33+
34+
This section contains an example of what that data would look like for each event.
35+
36+
### Microsoft.MachineLearningServices.ModelRegistered event
37+
38+
```json
39+
[{
40+
"topic": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace-name}",
41+
"subject": "models/sklearn_regression_model:20",
42+
"eventType": "Microsoft.MachineLearningServices.ModelRegistered",
43+
"eventTime": "2017-06-26T18:41:00.9584103Z",
44+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
45+
"data": {
46+
"ModelName": "sklearn_regression_model",
47+
"ModelVersion": 20,
48+
"ModelTags": {
49+
"area": "diabetes",
50+
"type": "regression"
51+
},
52+
"ModelProperties": {
53+
"type": "test"
54+
}
55+
},
56+
"dataVersion": "",
57+
"metadataVersion": "1"
58+
}]
59+
```
60+
61+
### Microsoft.MachineLearningServices.ModelDeployed event
62+
63+
```json
64+
[{
65+
"topic": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace-name}",
66+
"subject": "endpoints/my-sklearn-service",
67+
"eventType": "Microsoft.MachineLearningServices.ModelDeployed",
68+
"eventTime": "2017-06-26T18:41:00.9584103Z",
69+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
70+
"data": {
71+
"ServiceName": "my-sklearn-service",
72+
"ServiceComputeType": "ACI",
73+
"ModelIds": "sklearn_regression_model:1,sklearn_regression_model:2",
74+
"ServiceTags": {
75+
"area": "diabetes",
76+
"type": "regression"
77+
},
78+
"ServiceProperties": {
79+
"type": "test"
80+
}
81+
},
82+
"dataVersion": "",
83+
"metadataVersion": "1"
84+
}]
85+
```
86+
87+
### Microsoft.MachineLearningServices.RunCompleted event
88+
89+
```json
90+
[{
91+
"topic": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace-name}",
92+
"subject": "experiments/0fa9dfaa-cba3-4fa7-b590-23e48548f5c1/runs/AutoML_ad912b2d-6467-4f32-a616-dbe4af6dd8fc_5",
93+
"eventType": "Microsoft.MachineLearningServices.RunCompleted",
94+
"eventTime": "2017-06-26T18:41:00.9584103Z",
95+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
96+
"data": {
97+
"ExperimentId": "0fa9dfaa-cba3-4fa7-b590-23e48548f5c1",
98+
"ExperimentName": "automl-local-regression",
99+
"RunId": "AutoML_ad912b2d-6467-4f32-a616-dbe4af6dd8fc_5",
100+
"RunType": null,
101+
"RunTags": {},
102+
"RunProperties": {
103+
"runTemplate": "automl_child",
104+
"pipeline_id": "5adc0a4fe02504a586f09a4fcbb241f9a4012062",
105+
"pipeline_spec": "{\"objects\": [{\"class_name\": \"StandardScaler\", \"module\": \"sklearn.preprocessing\", \"param_args\": [], \"param_kwargs\": {\"with_mean\": true, \"with_std\": false}, \"prepared_kwargs\": {}, \"spec_class\": \"preproc\"}, {\"class_name\": \"LassoLars\", \"module\": \"sklearn.linear_model\", \"param_args\": [], \"param_kwargs\": {\"alpha\": 0.001, \"normalize\": true}, \"prepared_kwargs\": {}, \"spec_class\": \"sklearn\"}], \"pipeline_id\": \"5adc0a4fe02504a586f09a4fcbb241f9a4012062\"}",
106+
"training_percent": "100",
107+
"predicted_cost": "0.062226144097381045",
108+
"iteration": "5",
109+
"run_template": "automl_child",
110+
"run_preprocessor": "StandardScalerWrapper",
111+
"run_algorithm": "LassoLars",
112+
"conda_env_data_location": "aml://artifact/ExperimentRun/dcid.AutoML_ad912b2d-6467-4f32-a616-dbe4af6dd8fc_5/outputs/conda_env_v_1_0_0.yml",
113+
"model_name": "AutoMLad912b2d65",
114+
"scoring_data_location": "aml://artifact/ExperimentRun/dcid.AutoML_ad912b2d-6467-4f32-a616-dbe4af6dd8fc_5/outputs/scoring_file_v_1_0_0.py",
115+
"model_data_location": "aml://artifact/ExperimentRun/dcid.AutoML_ad912b2d-6467-4f32-a616-dbe4af6dd8fc_5/outputs/model.pkl"
116+
}
117+
},
118+
"dataVersion": "",
119+
"metadataVersion": "1"
120+
}]
121+
```
122+
123+
### Microsoft.MachineLearningServices.DatasetDriftDetected event
124+
125+
```json
126+
[{
127+
"topic": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace-name}",
128+
"subject": "datadrifts/{}/runs/{}",
129+
"eventType": "Microsoft.MachineLearningServices.DatasetDriftDetected",
130+
"eventTime": "2017-06-26T18:41:00.9584103Z",
131+
"id": "831e1650-001e-001b-66ab-eeb76e069631",
132+
"data": {
133+
"DataDriftId": "01d29aa4-e6a4-470a-9ef3-66660d21f8ef",
134+
"DataDriftName": "myDriftMonitor",
135+
"RunId": "01d29aa4-e6a4-470a-9ef3-66660d21f8ef_1571590300380",
136+
"BaseDatasetId": "3c56d136-0f64-4657-a0e8-5162089a88a3",
137+
"TargetDatasetId": "d7e74d2e-c972-4266-b5fb-6c9c182d2a74",
138+
"DriftCoefficient": 0.83503490684792081,
139+
"StartTime": "2019-07-04T00:00:00+00:00",
140+
"EndTime": "2019-07-05T00:00:00+00:00"
141+
},
142+
"dataVersion": "",
143+
"metadataVersion": "1"
144+
}]
145+
```
146+
147+
## Event properties
148+
149+
An event has the following top-level data:
150+
151+
| Property | Type | Description |
152+
| -------- | ---- | ----------- |
153+
| topic | string | Full resource path to the event source. This field is not writeable. Event Grid provides this value. |
154+
| subject | string | Publisher-defined path to the event subject. |
155+
| eventType | string | One of the registered event types for this event source. |
156+
| eventTime | string | The time the event is generated based on the provider's UTC time. |
157+
| id | string | Unique identifier for the event. |
158+
| data | object | Blob storage event data. |
159+
| dataVersion | string | The schema version of the data object. The publisher defines the schema version. |
160+
| metadataVersion | string | The schema version of the event metadata. Event Grid defines the schema of the top-level properties. Event Grid provides this value. |
161+
162+
The data object has the following properties for each event type:
163+
164+
### Microsoft.MachineLearningServices.ModelRegistered
165+
166+
| Property | Type | Description |
167+
| -------- | ---- | ----------- |
168+
| ModelName | string | The name of the model that was registered. |
169+
| ModelVersion | int | The version of the model that was registered. |
170+
| ModelTags | object | The tags of the model that was registered. |
171+
| ModelProperties | object | The properties of the model that was registered. |
172+
173+
### Microsoft.MachineLearningServices.ModelDeployed
174+
175+
| Property | Type | Description |
176+
| -------- | ---- | ----------- |
177+
| ServiceName | string | The name of the deployed service. |
178+
| ServiceComputeType | string | The compute type (e.g. ACI, AKS) of the deployed service. |
179+
| ModelIds | string | A common separated list of model IDs. The IDs of the models deployed in the service. |
180+
| ServiceTags | object | The tags of the deployed service. |
181+
| ServiceProperties | object | The properties of the deployed service. |
182+
183+
### Microsoft.MachineLearningServices.RunCompleted
184+
185+
| Property | Type | Description |
186+
| -------- | ---- | ----------- |
187+
| ExperimentId | string | The ID of the experiment that the run belongs to. |
188+
| ExperimentName | string | The name of the experiment that the run belongs to. |
189+
| RunId | string | The ID of the Run that was completed. |
190+
| RunType | string | The Run Type of the completed Run. |
191+
| RunTags | object | The tags of the completed Run. |
192+
| RunProperties | object | The properties of the completed Run. |
193+
194+
### Microsoft.MachineLearningServices.DatasetDriftDetected
195+
196+
| Property | Type | Description |
197+
| -------- | ---- | ----------- |
198+
| DataDriftId | string | The ID of the data drift monitor that triggered the event. |
199+
| DataDriftName | string | The name of the data drift monitor that triggered the event. |
200+
| RunId | string | The ID of the Run that detected data drift. |
201+
| BaseDatasetId | string | The ID of the base Dataset used to detect drift. |
202+
| TargetDatasetId | string | The ID of the target Dataset used to detect drift. |
203+
| DriftCoefficient | double | The coefficient result that triggered the event. |
204+
| StartTime | datetime | The start time of the target dataset time series that resulted in drift detection. |
205+
| EndTime | datetime | The end time of the target dataset time series that resulted in drift detection. |
206+
207+
208+
## Next steps
209+
210+
* For an introduction to Azure Event Grid, see [What is Event Grid?](overview.md)
211+
* For more information about creating an Azure Event Grid subscription, see [Event Grid subscription schema](subscription-creation-schema.md)
212+
* For an introduction to using Azure Event Grid with Azure Machine Learning, see [Consume Azure Machine Learning events](/azure/machine-learning/service/concept-event-grid-integration)
213+
* For an example of using Azure Event Grid with Azure Machine Learning, see [Create event driven machine learning workflows](/azure/machine-learning/service/event-schema-machine-learning)

‎articles/event-grid/event-schema.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,7 @@ To learn about the properties in the data object, see the event source:
101101
* [Resource groups (management operations)](event-schema-resource-groups.md)
102102
* [Service Bus](event-schema-service-bus.md)
103103
* [Azure SignalR](event-schema-azure-signalr.md)
104+
* [Azure Machine Learning](event-schema-machine-learning.md)
104105

105106
For custom topics, the event publisher determines the data object. The top-level data should have the same fields as standard resource-defined events.
106107

‎articles/event-grid/event-sources.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,14 @@ Subscribe to Azure SignalR Service events to respond to client connection events
167167
| [How to send Azure SignalR Service events to Event Grid](../azure-signalr/signalr-howto-event-grid-integration.md) | Shows how to send Azure SignalR Service events to an application through Event Grid. |
168168
| [Event schema](event-schema-azure-signalr.md) | Shows fields in Azure SignalR Service events. |
169169

170+
## Azure Machine Learning
170171

172+
Subscribe to Azure Machine Learning workspace events to respond to model registration.
173+
174+
| Title | Description |
175+
| ----- | ----- |
176+
| [Consume Azure Machine Learning events](../machine-learning/service/concept-event-grid-integration.md) | Overview of integrating Azure Machine Learning with Event Grid. |
177+
| [Azure Event Grid event schema for Azure Machine Learning](event-schema-machine-learning.md) | Shows fields in the Azure Machine Learning events. |
171178

172179
## Next steps
173180

‎articles/event-grid/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,8 @@
176176
href: event-schema-app-configuration.md
177177
- name: Azure SignalR
178178
href: event-schema-azure-signalr.md
179+
- name: Azure Machine Learning
180+
href: event-schema-machine-learning.md
179181
- name: Template resources
180182
items:
181183
- name: Event subscriptions

‎articles/machine-learning/service/concept-event-grid-integration.md

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
---
2+
title: 'Consume Azure Machine Learning events'
3+
titleSuffix: Azure Machine Learning
4+
description: In this article, learn how to use Azure Event Grid to subscribe, react, and unsubscribe to events generated by Azure Machine Learning.
5+
services: machine-learning
6+
ms.service: machine-learning
7+
ms.subservice: core
8+
ms.topic: conceptual
9+
ms.author: andyxu
10+
author: gogowings
11+
ms.date: 11/04/2019
12+
---
13+
14+
# Consume Azure Machine Learning events (Preview)
15+
16+
Azure Machine Learning manages the entire lifecycle of machine learning process, including model training, model deployment, and monitoring. Azure Machine Learning events allow applications to react to events during the machine learning lifecycle, such as the completion of training runs, the registration and deployment of models, and the detection of data drift, by using modern serverless architectures.
17+
18+
These events are published through [Azure Event Grid](https://azure.microsoft.com/services/event-grid/). Using Azure portal, Powershell or Azure CLI, customers can easily subscribe events by [specifying one or more event types, and filtering conditions](/azure/event-grid/event-filtering). Customers also have choice to build a wide range of event handlers such as Azure Functions, Azure Logic Apps, or generic webhooks. Azure Machine Learning, along with Azure Event Grid, provides a high available, reliable, and fault-tolerant event delivery platform for you to build event driven applications.
19+
20+
For information on using Azure Machine Learning with Event Grid, see [Create Event Driven Machine Learning Workflows (Preview)](how-to-use-event-grid.md).
21+
22+
## The event model
23+
24+
Azure Event Grid reads events from sources, such as Azure Machine Learning and other Azure services. These events are then sent to event handlers such as Azure Event Hubs, Azure Functions, Logic Apps, and others. The following diagram shows how Event Grid connects sources and handlers, but is not a comprehensive list of supported integrations.
25+
26+
![Azure Event Grid functional model](./media/concept-event-grid-integration/azure-event-grid-functional-model.png)
27+
28+
For more information on event sources and event handlers, see [What is Event Grid?](/azure/event-grid/overview).
29+
30+
## Azure Machine Learning event types
31+
32+
Azure Machine Learning provides events in the various points of machine learning lifecycle:
33+
34+
| Event type | Description |
35+
| ---------- | ----------- |
36+
| `Microsoft.MachineLearningServices.RunCompleted` | Raised when a machine learning experiment run is completed |
37+
| `Microsoft.MachineLearningServices.ModelRegistered` | Raised when a machine learning model is registered in the workspace |
38+
| `Microsoft.MachineLearningServices.ModelDeployed` | Raised when a deployment of inference service with one or more models is completed |
39+
| `Microsoft.MachineLearningServices.DatasetDriftDetected` | Raised when a data drift detection job for two datasets is completed |
40+
41+
## Subscribe to Machine Learning events
42+
43+
Subscriptions for Azure Machine Learning events are protected by role-based access control (RBAC). Only [contributor or owner](how-to-assign-roles.md#default-roles) of a workspace can create, update, and delete event subscriptions.
44+
45+
Event subscriptions can be filtered based on a variety of conditions. Filters can be applied to event subscriptions either during the [creation](/cli/azure/eventgrid/event-subscription?view=azure-cli-latest) of the event subscription or [at a later time](/cli/azure/eventgrid/event-subscription?view=azure-cli-latest).
46+
47+
### Filter by event type
48+
An event subscription can specify one or more Azure Machine Learning event types.
49+
50+
### Filter by event subject
51+
Azure Event Grid supports subject filters based on __begins with__ and __ends with__ matches, so that events with a matching subject are delivered to the subscriber. Different machine learning events have different subject format.
52+
53+
| Event type | Subject format | Sample subject |
54+
| ---------- | ----------- | ----------- |
55+
| `Microsoft.MachineLearningServices.RunCompleted` | `experiments/{ExperimentId}/runs/{RunId}` | `experiments/b1d7966c-f73a-4c68-b846-992ace89551f/runs/my_exp1_1554835758_38dbaa94` |
56+
| `Microsoft.MachineLearningServices.ModelRegistered` | `models/{modelName}:{modelVersion}` | `models/sklearn_regression_model:3` |
57+
| `Microsoft.MachineLearningServices.ModelDeployed` | `endpoints/{serviceId}` | `endpoints/my_sklearn_aks` |
58+
| `Microsoft.MachineLearningServices.DatasetDriftDetected` | `datadrift/{data.DataDriftId}/run/{data.RunId}` | `datadrift/4e694bf5-712e-4e40-b06a-d2a2755212d4/run/my_driftrun1_1550564444_fbbcdc0f` |
59+
60+
### Advanced filtering
61+
62+
Azure Event Grid also supports advanced filtering based on published event schema. Azure Machine Learning event schema details can be found in [Azure Event Grid event schema for Azure Machine Learning](../../event-grid/event-schema-machine-learning.md).
63+
64+
Some sample advanced filterings you can perform include:
65+
66+
* For `Microsoft.MachineLearningServices.ModelRegistered` event, to filter model's tag value:
67+
68+
```
69+
--advanced-filter data.ModelTags.key1 StringIn ('value1')
70+
```
71+
72+
To learn more about how to apply filters, see [Filter events for Event Grid](https://docs.microsoft.com/azure/event-grid/how-to-filter-events).
73+
74+
## Consume Machine Learning events
75+
76+
Applications that handle Machine Learning events should follow a few recommended practices:
77+
78+
> [!div class="checklist"]
79+
> * As multiple subscriptions can be configured to route events to the same event handler, it is important not to assume events are from a particular source, but to check the topic of the message to ensure that it comes from the storage account you are expecting.
80+
> * Similarly, check that the eventType is one you are prepared to process, and do not assume that all events you receive will be the types you expect.
81+
> * As messages can arrive out of order and after some delay, use the etag fields to understand if your information about objects is still up-to-date. Also, use the sequencer fields to understand the order of events on any particular object.
82+
> * Ignore fields you don't understand. This practice will help keep you resilient to new features that might be added in the future.
83+
> * Failed or cancelled Azure Machine Learning operations will not trigger an event. For example, if a model deployment fails Microsoft.MachineLearningServices.ModelDeployed won't be triggered. Consider such failure mode when design your applications. You can always use Azure Machine Learning SDK, CLI or portal to check the status of an operation and understand the detailed failure reasons.
84+
85+
Azure Event Grid allows customers to build de-coupled message handlers, which can be triggered by Azure Machine Learning events. Some notable examples of message handlers are:
86+
* Azure Functions
87+
* Azure Logic Apps
88+
* Azure Event Hubs
89+
* Azure Data Factory Pipeline
90+
* Generic webhooks, which may be hosted on the Azure platform or elsewhere
91+
92+
## Next steps
93+
94+
Learn more about Event Grid and give Azure Machine Learning events a try:
95+
96+
- [About Event Grid](../../event-grid/overview.md)
97+
- [Azure Event Grid event schema for Azure Machine Learning](../../event-grid/event-schema-machine-learning.md)
98+
- [Create event driven workflows with Azure Machine Learning](how-to-use-event-grid.md)

0 commit comments

Comments
 (0)