You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: This article describes four windowing functions (tumbling, hopping, sliding, session) that are used in Azure Stream Analytics jobs.
4
4
ms.service: stream-analytics
5
5
ms.topic: conceptual
6
-
ms.date: 08/29/2022
6
+
ms.date: 01/22/2024
7
7
---
8
8
# Introduction to Stream Analytics windowing functions
9
9
10
10
In time-streaming scenarios, performing operations on the data contained in temporal windows is a common pattern. Stream Analytics has native support for windowing functions, enabling developers to author complex stream processing jobs with minimal effort.
11
11
12
-
There are five kinds of temporal windows to choose from: [**Tumbling**](/stream-analytics-query/tumbling-window-azure-stream-analytics), [**Hopping**](/stream-analytics-query/hopping-window-azure-stream-analytics), [**Sliding**](/stream-analytics-query/sliding-window-azure-stream-analytics), [**Session**](/stream-analytics-query/session-window-azure-stream-analytics), and [**Snapshot**](/stream-analytics-query/snapshot-window-azure-stream-analytics) windows. You use the window functions in the [**GROUP BY**](/stream-analytics-query/group-by-azure-stream-analytics) clause of the query syntax in your Stream Analytics jobs. You can also aggregate events over multiple windows using the [**Windows()** function](/stream-analytics-query/windows-azure-stream-analytics).
12
+
There are five kinds of temporal windows to choose from:
13
13
14
-
All the [windowing](/stream-analytics-query/windowing-azure-stream-analytics) operations output results at the **end** of the window. Note that when you start a stream analytics job, you can specify the *Job output start time* and the system will automatically fetch previous events in the incoming streams to output the first window at the specified time; for example when you start with the *Now* option, it will start to emit data immediately.
15
-
The output of the window will be a single event based on the aggregate function used. The output event will have the time stamp of the end of the window and all window functions are defined with a fixed length.
You use the window functions in the [**GROUP BY**](/stream-analytics-query/group-by-azure-stream-analytics) clause of the query syntax in your Stream Analytics jobs. You can also aggregate events over multiple windows using the [**Windows()** function](/stream-analytics-query/windows-azure-stream-analytics).
21
+
22
+
All the [windowing](/stream-analytics-query/windowing-azure-stream-analytics) operations output results at the **end** of the window. When you start a stream analytics job, you can specify the *Job output start time*, and the system automatically fetches previous events in the incoming streams to output the first window at the specified time; for example, when you start with the *Now* option, it starts to emit data immediately. The output of the window will be a single event based on the aggregate function used. The output event has the time stamp of the end of the window and all window functions are defined with a fixed length.
23
+
24
+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-conceptual.png" alt-text="Diagram that shows the concept of Stream Analytics window functions.":::
18
25
19
26
## Tumbling window
20
27
21
-
[**Tumbling**](/stream-analytics-query/tumbling-window-azure-stream-analytics) window functions are used to segment a data stream into distinct time segments and perform a function against them, such as the example below. The key differentiators of a Tumbling window are that they repeat, do not overlap, and an event cannot belong to more than one tumbling window.
28
+
Use [**Tumbling**](/stream-analytics-query/tumbling-window-azure-stream-analytics) window functions to segment a data stream into distinct time segments, and perform a function against them.
- An event can't belong to more than one tumbling window.
24
35
25
-
With the following input data (illustrated above):
36
+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-tumbling-intro.png" alt-text="Diagram that shows an example Stream Analytics tumbling window.":::
37
+
38
+
Here's the input data for the example:
26
39
27
40
|Stamp|CreatedAt|TimeZone|
28
41
|-|-|-|
@@ -31,15 +44,15 @@ With the following input data (illustrated above):
31
44
|4|2021-10-26T10:15:06|PST|
32
45
|...|...|...|
33
46
34
-
The following query:
47
+
Here's the sample query:
35
48
36
49
```SQL
37
50
SELECTSystem.Timestamp() as WindowEndTime, TimeZone, COUNT(*) AS Count
38
51
FROM TwitterStream TIMESTAMP BY CreatedAt
39
52
GROUP BY TimeZone, TumblingWindow(second,10)
40
53
```
41
54
42
-
Will return:
55
+
Here's the sample output:
43
56
44
57
|WindowEndTime|TimeZone|Count|
45
58
|-|-|-|
@@ -49,11 +62,11 @@ Will return:
49
62
50
63
## Hopping window
51
64
52
-
[**Hopping**](/stream-analytics-query/hopping-window-azure-stream-analytics) window functions hop forward in time by a fixed period. It may be easy to think of them as Tumbling windows that can overlap and be emitted more often than the window size. Events can belong to more than one Hopping window result set. To make a Hopping window the same as a Tumbling window, specify the hop size to be the same as the window size.
65
+
[**Hopping**](/stream-analytics-query/hopping-window-azure-stream-analytics) window functions hop forward in time by a fixed period. It might be easy to think of them as tumbling windows that can overlap and be emitted more often than the window size. Events can belong to more than one Hopping window result set. To make a Hopping window the same as a Tumbling window, specify the hop size to be the same as the window size.
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-hopping-intro.png" alt-text="Diagram that shows an example of the hopping window.":::
55
68
56
-
With the following input data (illustrated above):
69
+
Here's the sample data:
57
70
58
71
|Stamp|CreatedAt|Topic|
59
72
|-|-|-|
@@ -62,15 +75,15 @@ With the following input data (illustrated above):
62
75
|4|2021-10-26T10:15:06|Streaming|
63
76
|...|...|...|
64
77
65
-
The following query:
78
+
Here's the sample query:
66
79
67
80
```SQL
68
81
SELECTSystem.Timestamp() as WindowEndTime, Topic, COUNT(*) AS Count
69
82
FROM TwitterStream TIMESTAMP BY CreatedAt
70
83
GROUP BY Topic, HoppingWindow(second,10,5)
71
84
```
72
85
73
-
Will return:
86
+
Here's the sample output:
74
87
75
88
|WindowEndTime|Topic|Count|
76
89
|-|-|-|
@@ -82,11 +95,11 @@ Will return:
82
95
83
96
## Sliding window
84
97
85
-
[**Sliding**](/stream-analytics-query/sliding-window-azure-stream-analytics) windows, unlike Tumbling or Hopping windows, output events only for points in time when the content of the window actually changes. In other words, when an event enters or exits the window. So, every window has at least one event. Similar to Hopping windows, events can belong to more than one sliding window.
98
+
[**Sliding**](/stream-analytics-query/sliding-window-azure-stream-analytics) windows, unlike tumbling or hopping windows, output events only for points in time when the content of the window actually changes. In other words, when an event enters or exits the window. So, every window has at least one event. Similar to hopping windows, events can belong to more than one sliding window.
86
99
87
-

100
+
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-sliding-intro.png" alt-text="Diagram that shows an example of a sliding window.":::
88
101
89
-
With the following input data (illustrated above):
102
+
Here's the sample input data:
90
103
91
104
|Stamp|CreatedAt|Topic|
92
105
|-|-|-|
@@ -96,7 +109,7 @@ With the following input data (illustrated above):
96
109
|7|2021-10-26T10:15:15|Streaming|
97
110
|8|2021-10-26T10:15:27|Streaming|
98
111
99
-
The following query:
112
+
Here's the sample query:
100
113
101
114
```SQL
102
115
SELECTSystem.Timestamp() as WindowEndTime, Topic, COUNT(*) AS Count
@@ -105,7 +118,7 @@ GROUP BY Topic, SlidingWindow(second,10)
105
118
HAVINGCOUNT(*) >=3
106
119
```
107
120
108
-
Will return:
121
+
Output:
109
122
110
123
|WindowEndTime|Topic|Count|
111
124
|-|-|-|
@@ -114,17 +127,21 @@ Will return:
114
127
115
128
## Session window
116
129
117
-
[**Session**](/stream-analytics-query/session-window-azure-stream-analytics) window functions group events that arrive at similar times, filtering out periods of time where there is no data. It has three main parameters: timeout, maximum duration, and partitioning key (optional).
130
+
[**Session**](/stream-analytics-query/session-window-azure-stream-analytics) window functions group events that arrive at similar times, filtering out periods of time where there's no data. It has three main parameters:
:::image type="content" source="media/stream-analytics-window-functions/stream-analytics-window-functions-session-intro.png" alt-text="Diagram that shows a sample Stream Analytics session window.":::
120
137
121
138
A session window begins when the first event occurs. If another event occurs within the specified timeout from the last ingested event, then the window extends to include the new event. Otherwise if no events occur within the timeout, then the window is closed at the timeout.
122
139
123
-
If events keep occurring within the specified timeout, the session window will keep extending until maximum duration is reached. The maximum duration checking intervals are set to be the same size as the specified max duration. For example, if the max duration is 10, then the checks on if the window exceed maximum duration will happen at t = 0, 10, 20, 30, etc.
140
+
If events keep occurring within the specified timeout, the session window keeps extending until maximum duration is reached. The maximum duration checking intervals are set to be the same size as the specified max duration. For example, if the max duration is 10, then the checks on if the window exceeds maximum duration happen at t = 0, 10, 20, 30, etc.
124
141
125
142
When a partition key is provided, the events are grouped together by the key and session window is applied to each group independently. This partitioning is useful for cases where you need different session windows for different users or devices.
126
143
127
-
With the following input data (illustrated above):
144
+
Here's the sample input data:
128
145
129
146
|Stamp|CreatedAt|Topic|
130
147
|-|-|-|
@@ -133,15 +150,15 @@ With the following input data (illustrated above):
133
150
|3|2021-10-26T10:15:13|Streaming|
134
151
|...|...|...|
135
152
136
-
The following query:
153
+
Here's the sample query:
137
154
138
155
```SQL
139
156
SELECTSystem.Timestamp() as WindowEndTime, Topic, COUNT(*) AS Count
140
157
FROM TwitterStream TIMESTAMP BY CreatedAt
141
158
GROUP BY Topic, SessionWindow(second,5,10)
142
159
```
143
160
144
-
Will return:
161
+
Output:
145
162
146
163
|WindowEndTime|Topic|Count|
147
164
|-|-|-|
@@ -154,9 +171,9 @@ Will return:
154
171
155
172
[**Snapshot**](/stream-analytics-query/snapshot-window-azure-stream-analytics) windows group events that have the same timestamp. Unlike other windowing types, which require a specific window function (such as [SessionWindow()](/stream-analytics-query/session-window-azure-stream-analytics)), you can apply a snapshot window by adding System.Timestamp() to the GROUP BY clause.
0 commit comments