Skip to content

Commit 1d380a8

Browse files
authored
Merge pull request #263854 from spelluru/asafreshness0122
Freshness review
2 parents 25e7982 + b40f08c commit 1d380a8

File tree

1 file changed

+49
-30
lines changed

1 file changed

+49
-30
lines changed

articles/stream-analytics/stream-analytics-window-functions.md

Lines changed: 49 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -3,26 +3,39 @@ title: Introduction to Azure Stream Analytics windowing functions
33
description: This article describes four windowing functions (tumbling, hopping, sliding, session) that are used in Azure Stream Analytics jobs.
44
ms.service: stream-analytics
55
ms.topic: conceptual
6-
ms.date: 08/29/2022
6+
ms.date: 01/22/2024
77
---
88
# Introduction to Stream Analytics windowing functions
99

1010
In time-streaming scenarios, performing operations on the data contained in temporal windows is a common pattern. Stream Analytics has native support for windowing functions, enabling developers to author complex stream processing jobs with minimal effort.
1111

12-
There are five kinds of temporal windows to choose from: [**Tumbling**](/stream-analytics-query/tumbling-window-azure-stream-analytics), [**Hopping**](/stream-analytics-query/hopping-window-azure-stream-analytics), [**Sliding**](/stream-analytics-query/sliding-window-azure-stream-analytics), [**Session**](/stream-analytics-query/session-window-azure-stream-analytics), and [**Snapshot**](/stream-analytics-query/snapshot-window-azure-stream-analytics) windows. You use the window functions in the [**GROUP BY**](/stream-analytics-query/group-by-azure-stream-analytics) clause of the query syntax in your Stream Analytics jobs. You can also aggregate events over multiple windows using the [**Windows()** function](/stream-analytics-query/windows-azure-stream-analytics).
12+
There are five kinds of temporal windows to choose from:
1313

14-
All the [windowing](/stream-analytics-query/windowing-azure-stream-analytics) operations output results at the **end** of the window. Note that when you start a stream analytics job, you can specify the *Job output start time* and the system will automatically fetch previous events in the incoming streams to output the first window at the specified time; for example when you start with the *Now* option, it will start to emit data immediately.
15-
The output of the window will be a single event based on the aggregate function used. The output event will have the time stamp of the end of the window and all window functions are defined with a fixed length.
14+
- [**Tumbling**](/stream-analytics-query/tumbling-window-azure-stream-analytics)
15+
- [**Hopping**](/stream-analytics-query/hopping-window-azure-stream-analytics)
16+
- [**Sliding**](/stream-analytics-query/sliding-window-azure-stream-analytics)
17+
- [**Session**](/stream-analytics-query/session-window-azure-stream-analytics)
18+
- [**Snapshot**](/stream-analytics-query/snapshot-window-azure-stream-analytics) windows.
1619

17-
![Stream Analytics window functions concepts](media/stream-analytics-window-functions/stream-analytics-window-functions-conceptual.png)
20+
You use the window functions in the [**GROUP BY**](/stream-analytics-query/group-by-azure-stream-analytics) clause of the query syntax in your Stream Analytics jobs. You can also aggregate events over multiple windows using the [**Windows()** function](/stream-analytics-query/windows-azure-stream-analytics).
21+
22+
All the [windowing](/stream-analytics-query/windowing-azure-stream-analytics) operations output results at the **end** of the window. When you start a stream analytics job, you can specify the *Job output start time*, and the system automatically fetches previous events in the incoming streams to output the first window at the specified time; for example, when you start with the *Now* option, it starts to emit data immediately. The output of the window will be a single event based on the aggregate function used. The output event has the time stamp of the end of the window and all window functions are defined with a fixed length.
23+
24+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-conceptual.png" alt-text="Diagram that shows the concept of Stream Analytics window functions.":::
1825

1926
## Tumbling window
2027

21-
[**Tumbling**](/stream-analytics-query/tumbling-window-azure-stream-analytics) window functions are used to segment a data stream into distinct time segments and perform a function against them, such as the example below. The key differentiators of a Tumbling window are that they repeat, do not overlap, and an event cannot belong to more than one tumbling window.
28+
Use [**Tumbling**](/stream-analytics-query/tumbling-window-azure-stream-analytics) window functions to segment a data stream into distinct time segments, and perform a function against them.
29+
30+
The key differentiators of a tumbling window are:
2231

23-
![Stream Analytics tumbling window](media/stream-analytics-window-functions/stream-analytics-window-functions-tumbling-intro.png)
32+
- They don't repeat.
33+
- They don't overlap.
34+
- An event can't belong to more than one tumbling window.
2435

25-
With the following input data (illustrated above):
36+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-tumbling-intro.png" alt-text="Diagram that shows an example Stream Analytics tumbling window.":::
37+
38+
Here's the input data for the example:
2639

2740
|Stamp|CreatedAt|TimeZone|
2841
|-|-|-|
@@ -31,15 +44,15 @@ With the following input data (illustrated above):
3144
|4|2021-10-26T10:15:06|PST|
3245
|...|...|...|
3346

34-
The following query:
47+
Here's the sample query:
3548

3649
```SQL
3750
SELECT System.Timestamp() as WindowEndTime, TimeZone, COUNT(*) AS Count
3851
FROM TwitterStream TIMESTAMP BY CreatedAt
3952
GROUP BY TimeZone, TumblingWindow(second,10)
4053
```
4154

42-
Will return:
55+
Here's the sample output:
4356

4457
|WindowEndTime|TimeZone|Count|
4558
|-|-|-|
@@ -49,11 +62,11 @@ Will return:
4962

5063
## Hopping window
5164

52-
[**Hopping**](/stream-analytics-query/hopping-window-azure-stream-analytics) window functions hop forward in time by a fixed period. It may be easy to think of them as Tumbling windows that can overlap and be emitted more often than the window size. Events can belong to more than one Hopping window result set. To make a Hopping window the same as a Tumbling window, specify the hop size to be the same as the window size.
65+
[**Hopping**](/stream-analytics-query/hopping-window-azure-stream-analytics) window functions hop forward in time by a fixed period. It might be easy to think of them as tumbling windows that can overlap and be emitted more often than the window size. Events can belong to more than one Hopping window result set. To make a Hopping window the same as a Tumbling window, specify the hop size to be the same as the window size.
5366

54-
![Stream Analytics hopping window](media/stream-analytics-window-functions/stream-analytics-window-functions-hopping-intro.png)
67+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-hopping-intro.png" alt-text="Diagram that shows an example of the hopping window.":::
5568

56-
With the following input data (illustrated above):
69+
Here's the sample data:
5770

5871
|Stamp|CreatedAt|Topic|
5972
|-|-|-|
@@ -62,15 +75,15 @@ With the following input data (illustrated above):
6275
|4|2021-10-26T10:15:06|Streaming|
6376
|...|...|...|
6477

65-
The following query:
78+
Here's the sample query:
6679

6780
```SQL
6881
SELECT System.Timestamp() as WindowEndTime, Topic, COUNT(*) AS Count
6982
FROM TwitterStream TIMESTAMP BY CreatedAt
7083
GROUP BY Topic, HoppingWindow(second,10,5)
7184
```
7285

73-
Will return:
86+
Here's the sample output:
7487

7588
|WindowEndTime|Topic|Count|
7689
|-|-|-|
@@ -82,11 +95,11 @@ Will return:
8295

8396
## Sliding window
8497

85-
[**Sliding**](/stream-analytics-query/sliding-window-azure-stream-analytics) windows, unlike Tumbling or Hopping windows, output events only for points in time when the content of the window actually changes. In other words, when an event enters or exits the window. So, every window has at least one event. Similar to Hopping windows, events can belong to more than one sliding window.
98+
[**Sliding**](/stream-analytics-query/sliding-window-azure-stream-analytics) windows, unlike tumbling or hopping windows, output events only for points in time when the content of the window actually changes. In other words, when an event enters or exits the window. So, every window has at least one event. Similar to hopping windows, events can belong to more than one sliding window.
8699

87-
![Stream Analytics 10 second sliding window](media/stream-analytics-window-functions/stream-analytics-window-functions-sliding-intro.png)
100+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-sliding-intro.png" alt-text="Diagram that shows an example of a sliding window.":::
88101

89-
With the following input data (illustrated above):
102+
Here's the sample input data:
90103

91104
|Stamp|CreatedAt|Topic|
92105
|-|-|-|
@@ -96,7 +109,7 @@ With the following input data (illustrated above):
96109
|7|2021-10-26T10:15:15|Streaming|
97110
|8|2021-10-26T10:15:27|Streaming|
98111

99-
The following query:
112+
Here's the sample query:
100113

101114
```SQL
102115
SELECT System.Timestamp() as WindowEndTime, Topic, COUNT(*) AS Count
@@ -105,7 +118,7 @@ GROUP BY Topic, SlidingWindow(second,10)
105118
HAVING COUNT(*) >=3
106119
```
107120

108-
Will return:
121+
Output:
109122

110123
|WindowEndTime|Topic|Count|
111124
|-|-|-|
@@ -114,17 +127,21 @@ Will return:
114127

115128
## Session window
116129

117-
[**Session**](/stream-analytics-query/session-window-azure-stream-analytics) window functions group events that arrive at similar times, filtering out periods of time where there is no data. It has three main parameters: timeout, maximum duration, and partitioning key (optional).
130+
[**Session**](/stream-analytics-query/session-window-azure-stream-analytics) window functions group events that arrive at similar times, filtering out periods of time where there's no data. It has three main parameters:
131+
132+
- Timeout
133+
- Maximum duration
134+
- Partitioning key (optional).
118135

119-
![Stream Analytics session window](media/stream-analytics-window-functions/stream-analytics-window-functions-session-intro.png)
136+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-session-intro.png" alt-text="Diagram that shows a sample Stream Analytics session window.":::
120137

121138
A session window begins when the first event occurs. If another event occurs within the specified timeout from the last ingested event, then the window extends to include the new event. Otherwise if no events occur within the timeout, then the window is closed at the timeout.
122139

123-
If events keep occurring within the specified timeout, the session window will keep extending until maximum duration is reached. The maximum duration checking intervals are set to be the same size as the specified max duration. For example, if the max duration is 10, then the checks on if the window exceed maximum duration will happen at t = 0, 10, 20, 30, etc.
140+
If events keep occurring within the specified timeout, the session window keeps extending until maximum duration is reached. The maximum duration checking intervals are set to be the same size as the specified max duration. For example, if the max duration is 10, then the checks on if the window exceeds maximum duration happen at t = 0, 10, 20, 30, etc.
124141

125142
When a partition key is provided, the events are grouped together by the key and session window is applied to each group independently. This partitioning is useful for cases where you need different session windows for different users or devices.
126143

127-
With the following input data (illustrated above):
144+
Here's the sample input data:
128145

129146
|Stamp|CreatedAt|Topic|
130147
|-|-|-|
@@ -133,15 +150,15 @@ With the following input data (illustrated above):
133150
|3|2021-10-26T10:15:13|Streaming|
134151
|...|...|...|
135152

136-
The following query:
153+
Here's the sample query:
137154

138155
```SQL
139156
SELECT System.Timestamp() as WindowEndTime, Topic, COUNT(*) AS Count
140157
FROM TwitterStream TIMESTAMP BY CreatedAt
141158
GROUP BY Topic, SessionWindow(second,5,10)
142159
```
143160

144-
Will return:
161+
Output:
145162

146163
|WindowEndTime|Topic|Count|
147164
|-|-|-|
@@ -154,9 +171,9 @@ Will return:
154171

155172
[**Snapshot**](/stream-analytics-query/snapshot-window-azure-stream-analytics) windows group events that have the same timestamp. Unlike other windowing types, which require a specific window function (such as [SessionWindow()](/stream-analytics-query/session-window-azure-stream-analytics)), you can apply a snapshot window by adding System.Timestamp() to the GROUP BY clause.
156173

157-
![Stream Analytics snapshot window](media/stream-analytics-window-functions/stream-analytics-window-functions-snapshot-intro.png)
174+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-snapshot-intro.png" alt-text="Diagram that shows a sample Steam Analytics snapshot window.":::
158175

159-
With the following input data (illustrated above):
176+
Here's the sample input data:
160177

161178
|Stamp|CreatedAt|Topic|
162179
|-|-|-|
@@ -165,15 +182,15 @@ With the following input data (illustrated above):
165182
|3|2021-10-26T10:15:04|Streaming|
166183
|...|...|...|
167184

168-
The following query:
185+
Here's the sample query:
169186

170187
```SQL
171188
SELECT System.Timestamp() as WindowEndTime, Topic, COUNT(*) AS Count
172189
FROM TwitterStream TIMESTAMP BY CreatedAt
173190
GROUP BY Topic, System.Timestamp()
174191
```
175192

176-
Will return:
193+
Here's the sample output:
177194

178195
|WindowEndTime|Topic|Count|
179196
|-|-|-|
@@ -184,6 +201,8 @@ Will return:
184201

185202
## Next steps
186203

204+
See the following articles:
205+
187206
* [Introduction to Azure Stream Analytics](stream-analytics-introduction.md)
188207
* [Get started using Azure Stream Analytics](stream-analytics-real-time-fraud-detection.md)
189208
* [Scale Azure Stream Analytics jobs](stream-analytics-scale-jobs.md)

0 commit comments

Comments
 (0)