Skip to content

Commit 31dd59a

Browse files
v-dirichardsv-dirichards
authored
Merge pull request #216622 from mrbullwinkle/mrb_10_31_2022_ad_streaming_inference
[Cognitive Services] [Anomaly Detector] Streaming inference
2 parents 5c38151 + db02d36 commit 31dd59a

File tree

2 files changed

+213
-19
lines changed

2 files changed

+213
-19
lines changed

‎articles/cognitive-services/Anomaly-Detector/How-to/streaming-inference.md

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
---
2+
title: Streaming inference with trained model
3+
titleSuffix: Azure Cognitive Services
4+
description: Streaming inference with trained model
5+
services: cognitive-services
6+
author: mrbullwinkle
7+
manager: nitinme
8+
ms.service: cognitive-services
9+
ms.subservice: anomaly-detector
10+
ms.topic: conceptual
11+
ms.date: 11/01/2022
12+
ms.author: mbullwin
13+
---
14+
15+
# Streaming inference with trained model
16+
17+
You could choose the batch inference API, or the streaming inference API for detection.
18+
19+
| Batch inference API | Streaming inference API |
20+
| ------------- | ---------------- |
21+
| More suitable for batch use cases when customers don’t need to get inference results immediately and want to detect anomalies and get results over a longer time period.| When customers want to get inference immediately and want to detect multivariate anomalies in real-time, this API is recommended. Also suitable for customers having difficulties conducting the previous compressing and uploading process for inference. |
22+
23+
|API Name| Method | Path | Description |
24+
| ------ | ---- | ----------- | ------ |
25+
|**Batch Inference**| POST | `{endpoint}`/anomalydetector/v1.1/multivariate/models/`{modelId}`: detect-batch | Trigger an asynchronous inference with `modelId` which works in a batch scenario |
26+
|**Get Batch Inference Results**| GET | `{endpoint}`/anomalydetector/v1.1/multivariate/detect-batch/`{resultId}` | Get batch inference results with `resultId` |
27+
|**Streaming Inference**| POST | `{endpoint}`/anomalydetector/v1.1/multivariate/models/`{modelId}`: detect-last | Trigger a synchronous inference with `modelId`, which works in a streaming scenario |
28+
29+
## Trigger a streaming inference API
30+
31+
### Request
32+
33+
With the synchronous API, you can get inference results point by point in real time, and no need for compressing and uploading task like for training and asynchronous inference. Here are some requirements for the synchronous API:
34+
* You need to put data in **JSON format** into the API request body.
35+
* Due to payload limitation, the size of inference data in the request body is limited, which support at most `2880` timestamps * `300` variables, and at least `1 sliding window length`.
36+
37+
You can submit a bunch of timestamps of multiple variables in JSON format in the request body, with an API call like this:
38+
39+
**{{endpoint}}/anomalydetector/v1.1/multivariate/models/{modelId}:detect-last**
40+
41+
A sample request:
42+
43+
```json
44+
{
45+
"variables": [
46+
{
47+
"variableName": "Variable_1",
48+
"timestamps": [
49+
"2021-01-01T00:00:00Z",
50+
"2021-01-01T00:01:00Z",
51+
"2021-01-01T00:02:00Z"
52+
//more timestamps
53+
],
54+
"values": [
55+
0.4551378545933972,
56+
0.7388603950488748,
57+
0.201088255984052
58+
//more values
59+
]
60+
},
61+
{
62+
"variableName": "Variable_2",
63+
"timestamps": [
64+
"2021-01-01T00:00:00Z",
65+
"2021-01-01T00:01:00Z",
66+
"2021-01-01T00:02:00Z"
67+
//more timestamps
68+
],
69+
"values": [
70+
0.9617871613964145,
71+
0.24903311574778408,
72+
0.4920561254118613
73+
//more values
74+
]
75+
},
76+
{
77+
"variableName": "Variable_3",
78+
"timestamps": [
79+
"2021-01-01T00:00:00Z",
80+
"2021-01-01T00:01:00Z",
81+
"2021-01-01T00:02:00Z"
82+
//more timestamps
83+
],
84+
"values": [
85+
0.4030756879437628,
86+
0.15526889968448554,
87+
0.36352226408981103
88+
//more values
89+
]
90+
}
91+
],
92+
"topContributorCount": 2
93+
}
94+
```
95+
96+
#### Required parameters
97+
98+
* **variableName**: This name should be exactly the same as in your training data.
99+
* **timestamps**: The length of the timestamps should be equal to **1 sliding window**, since every streaming inference call will use 1 sliding window to detect the last point in the sliding window.
100+
* **values**: The values of each variable in every timestamp that was inputted above.
101+
102+
#### Optional parameters
103+
104+
* **topContributorCount**: This is a number that you could specify N from **1 to 30**, which will give you the details of top N contributed variables in the anomaly results. For example, if you have 100 variables in the model, but you only care the top five contributed variables in detection results, then you should fill this field with 5. The default number is **10**.
105+
106+
### Response
107+
108+
A sample response:
109+
110+
```json
111+
{
112+
"variableStates": [
113+
{
114+
"variable": "series_0",
115+
"filledNARatio": 0.0,
116+
"effectiveCount": 1,
117+
"firstTimestamp": "2021-01-03T01:59:00Z",
118+
"lastTimestamp": "2021-01-03T01:59:00Z"
119+
},
120+
{
121+
"variable": "series_1",
122+
"filledNARatio": 0.0,
123+
"effectiveCount": 1,
124+
"firstTimestamp": "2021-01-03T01:59:00Z",
125+
"lastTimestamp": "2021-01-03T01:59:00Z"
126+
},
127+
{
128+
"variable": "series_2",
129+
"filledNARatio": 0.0,
130+
"effectiveCount": 1,
131+
"firstTimestamp": "2021-01-03T01:59:00Z",
132+
"lastTimestamp": "2021-01-03T01:59:00Z"
133+
},
134+
{
135+
"variable": "series_3",
136+
"filledNARatio": 0.0,
137+
"effectiveCount": 1,
138+
"firstTimestamp": "2021-01-03T01:59:00Z",
139+
"lastTimestamp": "2021-01-03T01:59:00Z"
140+
},
141+
{
142+
"variable": "series_4",
143+
"filledNARatio": 0.0,
144+
"effectiveCount": 1,
145+
"firstTimestamp": "2021-01-03T01:59:00Z",
146+
"lastTimestamp": "2021-01-03T01:59:00Z"
147+
}
148+
],
149+
"results": [
150+
{
151+
"timestamp": "2021-01-03T01:59:00Z",
152+
"value": {
153+
"isAnomaly": false,
154+
"severity": 0.0,
155+
"score": 0.2675322890281677,
156+
"interpretation": []
157+
},
158+
"errors": []
159+
}
160+
]
161+
}
162+
```
163+
164+
The response contains the result status, variable information, inference parameters, and inference results.
165+
166+
* **variableStates**: This lists the information of each variable in the inference request.
167+
* **setupInfo**: This is the request body submitted for this inference.
168+
* **results**: This contains the detection results. There are three typical types of detection results.
169+
170+
* **isAnomaly**: `false` indicates the current timestamp isn't an anomaly.`true` indicates an anomaly at the current timestamp.
171+
* `severity` indicates the relative severity of the anomaly and for abnormal data it's always greater than 0.
172+
* `score` is the raw output of the model on which the model makes a decision. `severity` is a derived value from `score`. Every data point has a `score`.
173+
174+
* **interpretation**: This field only appears when a timestamp is detected as anomalous, which contains `variables`, `contributionScore`, `correlationChanges`.
175+
176+
* **contributors**: This is a list containing the contribution score of each variable. Higher contribution scores indicate higher possibility of the root cause. This list is often used for interpreting anomalies and diagnosing the root causes.
177+
178+
* **correlationChanges**: This field only appears when a timestamp is detected as anomalous, which is included in interpretation. It contains `changedVariables` and `changedValues` that interpret which correlations between variables changed.
179+
180+
* **changedVariables**: This field will show which variables that have significant change in correlation with `variable`. The variables in this list are ranked by the extent of correlation changes.
181+
182+
> [!NOTE]
183+
> A common pitfall is taking all data points with `isAnomaly`=`true` as anomalies. That may end up with too many false positives.
184+
> You should use both `isAnomaly` and `severity` (or `score`) to sift out anomalies that are not severe and (optionally) use grouping to check the duration of the anomalies to suppress random noise.
185+
> Please refer to the [FAQ](../concepts/best-practices-multivariate.md#faq) in the best practices document for the difference between `severity` and `score`.
186+
187+
## Next steps
188+
189+
* [Multivariate Anomaly Detection reference architecture](../concepts/multivariate-architecture.md)
190+
* [Best practices of multivariate anomaly detection](../concepts/best-practices-multivariate.md)

‎articles/cognitive-services/Anomaly-Detector/toc.yml

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -29,31 +29,35 @@
2929
href: https://github.com/Azure-Samples/AnomalyDetector/tree/master/samples-multivariate
3030
- name: How-to guides
3131
items:
32-
- name: Identify anomalies in your data (univariate)
33-
href: how-to/identify-anomalies.md
34-
- name: Deploy Anomaly Detector (univariate)
32+
- name: Univariate Anomaly Detection
3533
items:
36-
- name: Deploy to IoT Edge
37-
href: how-to/deploy-anomaly-detection-on-iot-edge.md
38-
- name: Containers
39-
items:
40-
- name: Install and run containers
41-
displayName: docker, container, package, packaged app, swagger, open api, openapi, train, publish, az, acr, cli, tps, core, curl, try it now, import, pull
42-
href: anomaly-detector-container-howto.md
43-
- name: Configure containers
44-
href: anomaly-detector-container-configuration.md
45-
displayName: docker, container, alike, billing, log, mount, run, pull, port
46-
- name: Use container instances
47-
href: how-to/deploy-anomaly-detection-on-container-instances.md
48-
- name: All Cognitive Services containers documentation
49-
href: ../containers/index.yml
50-
maintainContext: true
51-
- name: Multivariate Anomaly Detector
34+
- name: Identify anomalies in your data
35+
href: how-to/identify-anomalies.md
36+
- name: Deploy Anomaly Detector (univariate)
37+
items:
38+
- name: Deploy to IoT Edge
39+
href: how-to/deploy-anomaly-detection-on-iot-edge.md
40+
- name: Containers
41+
items:
42+
- name: Install and run containers
43+
displayName: docker, container, package, packaged app, swagger, open api, openapi, train, publish, az, acr, cli, tps, core, curl, try it now, import, pull
44+
href: anomaly-detector-container-howto.md
45+
- name: Configure containers
46+
href: anomaly-detector-container-configuration.md
47+
displayName: docker, container, alike, billing, log, mount, run, pull, port
48+
- name: Use container instances
49+
href: how-to/deploy-anomaly-detection-on-container-instances.md
50+
- name: All Cognitive Services containers documentation
51+
href: ../containers/index.yml
52+
maintainContext: true
53+
- name: Multivariate Anomaly Detection
5254
items:
5355
- name: Train a model
5456
href: how-to/train-model.md
5557
- name: Batch inference
5658
href: how-to/batch-inference.md
59+
- name: Streaming inference
60+
href: how-to/streaming-inference.md
5761
- name: Enterprise readiness
5862
items:
5963
- name: Set up Virtual Networks

0 commit comments

Comments
 (0)