Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
6463013
Add documentation for TS source
kkrik-es Sep 9, 2025
8677ce0
add toc ref
kkrik-es Sep 9, 2025
49630b7
update nav
kkrik-es Sep 9, 2025
3beebc4
update
kkrik-es Sep 9, 2025
4104e1a
Merge branch 'main' into tsdb/ts-doc
marciw Sep 10, 2025
c860567
Apply suggestions from code review
kkrik-es Sep 11, 2025
5d23ffe
Update ts.md
kkrik-es Sep 11, 2025
4ec4daf
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 11, 2025
2feaffc
Apply suggestions from code review
kkrik-es Sep 12, 2025
d9e9451
update
kkrik-es Sep 12, 2025
fd54d14
add links to time-series functions
kkrik-es Sep 12, 2025
7dbd092
restore brief description
kkrik-es Sep 12, 2025
54f994b
fix formatting
kkrik-es Sep 12, 2025
b299ec4
remove links
kkrik-es Sep 12, 2025
0e56b47
Apply suggestions from code review
kkrik-es Sep 15, 2025
9b3bb28
Update ts.md
kkrik-es Sep 15, 2025
e452620
Update ts.md
kkrik-es Sep 16, 2025
bf342c2
Update ts.md
kkrik-es Sep 16, 2025
9b557ce
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 19, 2025
647534c
Merge branch 'refs/heads/main' into tsdb/ts-doc
kkrik-es Sep 19, 2025
8afa050
update
kkrik-es Sep 19, 2025
a83f082
Merge remote-tracking branch 'origin/tsdb/ts-doc' into tsdb/ts-doc
kkrik-es Sep 19, 2025
5cb5f45
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 23, 2025
086fece
mark over time functions as preview
kkrik-es Sep 24, 2025
a376340
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 24, 2025
020d26b
add time-series aggs
kkrik-es Sep 24, 2025
ef0611f
Merge remote-tracking branch 'origin/tsdb/ts-doc' into tsdb/ts-doc
kkrik-es Sep 24, 2025
cd670c1
list time-series aggs
kkrik-es Sep 24, 2025
51b2170
add missing
kkrik-es Sep 24, 2025
e499726
add toc
kkrik-es Sep 24, 2025
bd2f557
remove snapshot notes
kkrik-es Sep 24, 2025
a9cc818
add title
kkrik-es Sep 24, 2025
cd4ed00
sum_over_time example
kkrik-es Sep 24, 2025
8954b44
fix ts agg refs
kkrik-es Sep 24, 2025
1d25d54
add links to TS doc
kkrik-es Sep 24, 2025
8545d17
fix links to TS doc
kkrik-es Sep 24, 2025
c9a21d4
fix links to TS doc
kkrik-es Sep 24, 2025
9e192d9
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 24, 2025
4d7e593
remove note
kkrik-es Sep 24, 2025
f32b1ff
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 25, 2025
ee1df8b
Apply suggestions from code review
kkrik-es Sep 25, 2025
9276b86
Update ts.md
kkrik-es Sep 25, 2025
b0600a9
Apply suggestions from code review
kkrik-es Sep 25, 2025
aa587e8
Apply suggestions from code review
kkrik-es Sep 25, 2025
1f17af3
update function description
kkrik-es Sep 25, 2025
71fa400
Update docs/reference/query-languages/esql/_snippets/commands/layout/…
kkrik-es Sep 25, 2025
013a5e0
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 25, 2025
2b1832c
Apply suggestions from code review
kkrik-es Sep 26, 2025
59541ae
Merge branch 'main' into tsdb/ts-doc
kkrik-es Sep 26, 2025
a2706ea
fix from review
kkrik-es Sep 26, 2025
2921961
Update ts.md
kkrik-es Sep 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 93 additions & 7 deletions docs/reference/query-languages/esql/_snippets/commands/layout/ts.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
```yaml {applies_to}
serverless: ga
stack: ga
serverless: preview
stack: preview 9.2.0
```

The `TS` command is similar to the `FROM` source command,
but with two key differences: it targets only [time-series indices](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md)
and enables the use of time-series aggregation functions
with the [STATS](/reference/query-languages/esql/commands/stats-by.md) command.
**Brief description**

The `TS` source command is similar to the [`FROM`](/reference/query-languages/esql/commands/from.md)
source command, with the following key differences:

- Targets only [time series indices](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md)
- Enables the use of [time series aggregation functions](/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md) inside the
[STATS](/reference/query-languages/esql/commands/stats-by.md) command

**Syntax**

Expand All @@ -22,10 +26,92 @@ TS index_pattern [METADATA fields]
`fields`
: A comma-separated list of [metadata fields](/reference/query-languages/esql/esql-metadata-fields.md) to retrieve.

**Description**

The `TS` source command enables time series semantics and adds support for
[time series aggregation functions](/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md) to the `STATS` command, such as
[`AVG_OVER_TIME()`](/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md#esql-avg_over_time),
or [`RATE`](/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md#esql-rate).
These functions are implicitly evaluated per time series, then aggregated by group using a secondary aggregation
function. For example:
Comment on lines +35 to +36
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: maybe improve the wording a bit on this.

Suggested change
These functions are implicitly evaluated per time series, then aggregated by group using a secondary aggregation
function. For example:
These functions are implicitly evaluated per time series.
A secondary secondary aggregation function, in combination with a `BY` grouping expression, allows to group by a set of dimensions.
For example:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll stick with Marci's version.


```esql
TS metrics
| WHERE @timestamp >= now() - 1 hour
| STATS SUM(RATE(search_requests)) BY TBUCKET(1 hour), host
```

This query calculates the total rate of search requests (tracked by the `search_requests` counter) per host and hour. The `RATE()`
function is applied per time series in hourly buckets. These rates are summed for each
host and hourly bucket (since each host can map to multiple time series).

This paradigm—a pair of aggregation functions—is standard for time series
querying. For supported inner (time series) functions per
[metric type](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md#time-series-metric), refer to
[](/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md). These functions also
apply to downsampled data, with the same semantics as for raw data.

::::{note}
If a query is missing an inner (time series) aggregation function,
[`LAST_OVER_TIME()`](/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md#esql-last_over_time)
is assumed and used implicitly. For instance, the following two queries are
equivalent, returning the average of the last memory usage values per time series:

```esql
TS metrics | STATS AVG(memory_usage)

TS metrics | STATS AVG(LAST_OVER_TIME(memory_usage))
```

To calculate the average memory usage across per-time-series averages, use
the following query:

```esql
TS metrics | STATS AVG(AVG_OVER_TIME(memory_usage))
```
::::

Use regular (non-time-series)
[aggregation functions](/reference/query-languages/esql/functions-operators/aggregation-functions.md),
such as `SUM()`, as outer aggregation functions. Using a time series aggregation
in combination with an inner function causes an error. For example, the
Comment on lines +76 to +77
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
such as `SUM()`, as outer aggregation functions. Using a time series aggregation
in combination with an inner function causes an error. For example, the
such as `SUM()`, as outer aggregation functions.
Nesting time series aggregation functions causes an error.
For example, the

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm nesting is not very clear? Using them as inner functions can still be called "nesting".

following query is invalid:

```esql
TS metrics | STATS AVG_OVER_TIME(RATE(memory_usage))
```

::::{note}
A [time series](/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md)
aggregation function must be wrapped inside a
[regular](/reference/query-languages/esql/functions-operators/aggregation-functions.md)
aggregation function. For instance, the following query is invalid:

```esql
TS metrics | STATS RATE(search_requests)
```
::::

**Best practices**

- Avoid aggregating multiple metrics in the same query when those metrics have different dimensional cardinalities.
For example, in `STATS max(rate(foo)) + rate(bar))`, if `foo` and `bar` don't share the same dimension values, the rate
for one metric will be null for some dimension combinations. Because the + operator returns null when either input
is null, the entire result becomes null for those dimensions. Additionally, queries that aggregate a single metric
can filter out null values more efficiently.
- Use the `TS` command for aggregations on time series data, rather than `FROM`. The `FROM` command is still available
(for example, for listing document contents), but it's not optimized for procesing time series data and may produce
unexpected results.
- The `TS` command can't be combined with certain operations (such as
[`FORK`](/reference/query-languages/esql/commands/fork.md)) before the `STATS` command is applied. Once `STATS` is
applied, you can process the tabular output with any applicable ES|QL operations.
- Add a time range filter on `@timestamp` to limit the data volume scanned and improve query performance.

**Examples**

```esql
TS metrics
| STATS sum(last_over_time(memory_usage))
| WHERE @timestamp >= now() - 1 day
| STATS SUM(AVG_OVER_TIME(memory_usage)) BY host, TBUCKET(1 hour)
```

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading