Skip to content

Commit 96e9ef4

Browse files
DOC-5424 stash progress on compaction section
1 parent 644d71a commit 96e9ef4

File tree

1 file changed

+170
-66
lines changed
  • content/develop/data-types/timeseries

1 file changed

+170
-66
lines changed

content/develop/data-types/timeseries/_index.md

Lines changed: 170 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -19,92 +19,196 @@ weight: 150
1919
[![Discord](https://img.shields.io/discord/697882427875393627?style=flat-square)](https://discord.gg/KExRgMb)
2020
[![Github](https://img.shields.io/static/v1?label=&message=repository&color=5961FF&logo=github)](https://github.com/RedisTimeSeries/RedisTimeSeries/)
2121

22-
The Redis time series structure lets you store and query timestamped data points.
23-
24-
Redis time series is available in Redis Open Source, Redis Software, and Redis Cloud.
22+
The Redis time series data type lets you store real-valued data points
23+
along with the time they were collected. You can combine the values from a selection
24+
of time series and query them by time or value range. You can also compute
25+
aggregate functions of the data over periods of time and create new time series
26+
from the results. When you create a time series, you can specify a maximum
27+
retention period for the data, relative to the last reported timestamp, to
28+
prevent the time series from growing indefinitely.
29+
30+
Time series support very fast reads and writes, making them ideal for
31+
applications such as:
32+
33+
- Instrument data logging
34+
- System performance metrics
35+
- Financial market data
36+
- Internet of Things (IoT) sensor data
37+
- Smart metering
38+
- Quality of service (QoS) monitoring
39+
40+
Redis time series are available in Redis Open Source, Redis Software, and Redis Cloud.
2541
See
2642
[Install Redis Open Source]({{< relref "/operate/oss_and_stack/install/install-stack" >}}) or
2743
[Install Redis Enterprise]({{< relref "/operate/rs/installing-upgrading/install" >}})
2844
for full installation instructions.
2945

30-
## Features
31-
* High volume inserts, low latency reads
32-
* Query by start time and end-time
33-
* Aggregated queries (min, max, avg, sum, range, count, first, last, STD.P, STD.S, Var.P, Var.S, twa) for any time bucket
34-
* Configurable maximum retention period
35-
* Compaction for automatically updated aggregated timeseries
36-
* Secondary indexing for time series entries. Each time series has labels (field value pairs) which will allows to query by labels
37-
38-
## Creating a timeseries
39-
A new timeseries can be created with the [`TS.CREATE`]({{< relref "commands/ts.create/" >}}) command; for example, to create a timeseries named `sensor1` run the following:
40-
41-
```
42-
TS.CREATE sensor1
46+
## Creating a time series
47+
48+
You can create a new empty time series with the [`TS.CREATE`]({{< relref "commands/ts.create/" >}}) command, specifying a key name. If you use [`TS.ADD`]({{< relref "commands/ts.add/" >}}) to add data to a time series key that does not exist, it is automatically created.
49+
50+
```bash
51+
> TS.CREATE thermometer:1
52+
OK
53+
> TYPE thermometer:1
54+
TSDB-TYPE
55+
> TS.INFO thermometer:1
56+
1) totalSamples
57+
2) (integer) 0
58+
.
59+
.
60+
```
61+
62+
The timestamp for each data point is a 64-bit integer value. This is designed
63+
to support Unix timestamps, measured in milliseconds since the
64+
[Unix epoch](https://en.wikipedia.org/wiki/Unix_time). However, you can interpret
65+
the timestamps in any way you like (for example, as the number of days since a given start date).
66+
When you create a time series, you can specify a maximum retention period for the
67+
data, relative to the last reported timestamp. A retention period of `0` means
68+
the data does not expire.
69+
70+
```bash
71+
# Create a new time series with a first value of 10.8 (Celsius),
72+
# recorded on day 1, with a retention period of 100 days.
73+
> TS.ADD thermometer:2 1 10.8 RETENTION 100
74+
(integer) 1
75+
> TS.INFO thermometer:2
76+
.
77+
.
78+
9) retentionTime
79+
10) (integer) 100
80+
.
81+
.
82+
```
83+
84+
You can also add one or more *labels* to a time series when you create it. Labels
85+
are key-value pairs where the value can be a string or a number. You can use
86+
both the keys and values to select subsets of all the available time series
87+
for queries and aggregations.
88+
89+
```bash
90+
> TS.ADD thermometer:3 1 10.4 LABELS location UK type Mercury
91+
(integer) 1
92+
> TS.INFO thermometer:3
93+
1) totalSamples
94+
2) (integer) 1
95+
3) memoryUsage
96+
4) (integer) 5000
97+
.
98+
.
99+
19) labels
100+
20) 1) 1) "location"
101+
2) "UK"
102+
2) 1) "type"
103+
2) "Mercury"
104+
.
105+
.
43106
```
44107

45-
You can prevent your timeseries growing indefinitely by setting a maximum age for samples compared to the last event time (in milliseconds) with the `RETENTION` option. The default value for retention is `0`, which means the series will not be trimmed.
46-
47-
```
48-
TS.CREATE sensor1 RETENTION 2678400000
49-
```
50-
This will create a timeseries called `sensor1` and trim it to values of up to one month.
51-
52108

53109
## Adding data points
54-
For adding new data points to a timeseries we use the [`TS.ADD`]({{< relref "commands/ts.add/" >}}) command:
55-
56-
```
57-
TS.ADD key timestamp value
58-
```
59110

60-
The `timestamp` argument is the UNIX timestamp of the sample in milliseconds and `value` is the numeric data value of the sample.
111+
You can add individual data points with [`TS.ADD`]({{< relref "commands/ts.add/" >}}),
112+
but you can also use [`TS.MADD`]({{< relref "commands/ts.madd/" >}}) to add multiple data
113+
points to one or more time series in a single command. (Note that unlike `TS.ADD`, `TS.MADD`
114+
doesn't create any new time series if you specify keys that don't exist.) The return value
115+
is an array containing the number of samples in each time series after the operation.
116+
If you use the `*` character as the timestamp, Redis will record the current
117+
Unix time, as reported by the server's clock.
61118

62-
Example:
119+
```bash
120+
> TS.MADD thermometer:1 1 9.2 thermometer:1 2 9.9 thermometer:2 2 10.3
121+
1) (integer) 1
122+
2) (integer) 2
123+
3) (integer) 2
63124
```
64-
TS.ADD sensor1 1626434637914 26
65-
```
66-
67-
To **add a datapoint with the current timestamp** you can use a `*` instead of a specific timestamp:
68-
69-
```
70-
TS.ADD sensor1 * 26
71-
```
72-
73-
You can **append data points to multiple timeseries** at the same time with the [`TS.MADD`]({{< relref "commands/ts.madd/" >}}) command:
74-
```
75-
TS.MADD key timestamp value [key timestamp value ...]
76-
```
77-
78125

79126
## Deleting data points
80-
Data points between two timestamps (inclusive) can be deleted with the [`TS.DEL`]({{< relref "commands/ts.del/" >}}) command:
81-
```
82-
TS.DEL key fromTimestamp toTimestamp
83-
```
84-
Example:
85-
```
86-
TS.DEL sensor1 1000 2000
87-
```
88127

89-
To delete a single timestamp, use it as both the "from" and "to" timestamp:
90-
```
91-
TS.DEL sensor1 1000 1000
128+
Use [`TS.DEL`]({{< relref "commands/ts.del/" >}}) to delete data points
129+
that fall within a given timestamp range. The range is inclusive, meaning that
130+
samples whose timestamp equals the start or end of the range are deleted.
131+
If you want to delete a single timestamp, use it as both the start and end of the range.
132+
133+
```bash
134+
> TS.INFO thermometer:1
135+
1) totalSamples
136+
2) (integer) 2
137+
3) memoryUsage
138+
4) (integer) 4856
139+
5) firstTimestamp
140+
6) (integer) 1
141+
7) lastTimestamp
142+
8) (integer) 2
143+
.
144+
.
145+
> TS.ADD thermometer:1 3 9.7
146+
(integer) 3
147+
127.0.0.1:6379> TS.INFO thermometer:1
148+
1) totalSamples
149+
2) (integer) 3
150+
3) memoryUsage
151+
4) (integer) 4856
152+
5) firstTimestamp
153+
6) (integer) 1
154+
7) lastTimestamp
155+
8) (integer) 3
156+
.
157+
.
158+
> TS.DEL thermometer:1 1 2
159+
(integer) 2
160+
> TS.INFO thermometer:1
161+
1) totalSamples
162+
2) (integer) 1
163+
3) memoryUsage
164+
4) (integer) 4856
165+
5) firstTimestamp
166+
6) (integer) 3
167+
7) lastTimestamp
168+
8) (integer) 3
169+
.
170+
.
171+
> TS.DEL thermometer:1 3 3
172+
(integer) 1
173+
> TS.INFO thermometer:1
174+
1) totalSamples
175+
2) (integer) 0
176+
.
177+
.
92178
```
93179

94180
**Note:** When a sample is deleted, the data in all downsampled timeseries will be recalculated for the specific bucket. If part of the bucket has already been removed though, because it's outside of the retention period, we won't be able to recalculate the full bucket, so in those cases we will refuse the delete operation.
95181

96-
97-
## Labels
98-
Labels are key-value metadata we attach to data points, allowing us to group and filter. They can be either string or numeric values and are added to a timeseries on creation:
99-
100-
```
101-
TS.CREATE sensor1 LABELS region east
102-
```
103-
104-
105-
106182
## Compaction
107-
Another useful feature of Redis Time Series is compacting data by creating a rule for compaction ([`TS.CREATERULE`]({{< relref "commands/ts.createrule/" >}})). For example, if you have collected more than one billion data points in a day, you could aggregate the data by every minute in order to downsample it, thereby reducing the dataset size to 24 * 60 = 1,440 data points. You can choose one of the many available aggregation types in order to aggregate multiple data points from a certain minute into a single one. The currently supported aggregation types are: `avg, sum, min, max, range, count, first, last, std.p, std.s, var.p, var.s and twa`.
183+
184+
A time series can become large if samples are added very frequently. Instead
185+
of dealing with individual samples, it is sometimes useful to split the full
186+
time range of the series into equal-sized "buckets" and represent each
187+
bucket by an aggregate value, such as the average or maximum value.
188+
Reducing the number of data points in this way is known as *compaction*.
189+
190+
For example, if you expect to collect more than one billion data points in a day, you could aggregate the data using buckets of one minute. Since each bucket is represented by a single value, this compacts the dataset size to 1,440 data points (24 hours x 60 minutes = 1,440 minutes).
191+
192+
Use [`TS.CREATERULE`]({{< relref "commands/ts.createrule/" >}}) to create a
193+
194+
new
195+
compacted time series from an existing one, leaving the original series unchanged.
196+
Specify a duration for each bucket and an aggregation function to apply to each bucket.
197+
The available aggregation functions are:
198+
199+
- `avg`: Arithmetic mean of all values
200+
- `sum`: Sum of all values
201+
- `min`: Minimum value
202+
- `max`: Maximum value
203+
- `range`: Difference between the highest and the lowest value
204+
- `count`: Number of values
205+
- `first`: Value with lowest timestamp in the bucket
206+
- `last`: Value with highest timestamp in the bucket
207+
- `std.p`: Population standard deviation of the values
208+
- `std.s`: Sample standard deviation of the values
209+
- `var.p`: Population variance of the values
210+
- `var.s`: Sample variance of the values
211+
- `twa`: Time-weighted average over the bucket's timeframe (since RedisTimeSeries v1.8)
108212

109213
It's important to point out that there is no data rewriting on the original timeseries; the compaction happens in a new series, while the original one stays the same. In order to prevent the original timeseries from growing indefinitely, you can use the retention option, which will trim it down to a certain period of time.
110214

0 commit comments

Comments
 (0)