Add documentation for TS source #134373

kkrik-es · 2025-09-09T15:24:55Z

Covers how to apply time-series aggregations and the subtle differences with STATS under from.

Documentation is currently nested under TS command, but it covers how aggregations in STATS are affected. Another option is to move the time-series aggs in a dedicated section in STATS and split the documentation between TS and STATS.

github-actions · 2025-09-09T15:47:46Z

🔍 Preview links for changed docs

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

dnhatn

LGTM. Thanks Kostas!

dnhatn · 2025-09-09T21:55:43Z

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

+[metric type](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md#time-series-metric)
+include:
+
+ - `LAST_OVER_TIME()`: applies to gauges and counters


counter fields suport rate only.

Right, see #133295

marciw

@kkrik-es this is looking really good! Thank you for drafting it.

I know it's still a draft, but I gave it a quick read and made small suggestions. @leemthompo we'll definitely want your eyes on this too 🙏

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

Co-authored-by: Marci W <[email protected]>

kkrik-es · 2025-09-24T16:59:28Z

@leemthompo I've updated the time-series functions as well and included them here. Please take a look, happy to split this in more PRs if it helps.

marciw

Made some suggestions for clarity. Thank you!

Other docs typically don't hyphenate time series, even when they should 🙃 but I didn't mark it everywhere because it's not super critical.

@leemthompo over to you for a more essential review

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

marciw · 2025-09-24T21:49:35Z

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

+These functions are implicitly evaluated per per time-series, with their results
+then aggregated per grouping bucket using a secondary aggregation
+function. More concretely, consider the following query:


@leemthompo going for conciseness, but possibly too far?

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

marciw · 2025-09-24T23:34:22Z

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

+**Best practices**
+
+- Avoid mixing aggregation functions on different metrics in the same query, to
+  avoid interference between different time-series. For instance, if one metric


Claude's attempt to rewrite this 🤖 + some edits by me -- @kkrik-es WDYT? please check for accuracy!

Avoid aggregating multiple metrics in the same query when those metrics have different dimensional cardinalities. For example, in STATS max(rate(foo)) + rate(bar)), if foo and bar don't share the same dimension values, the rate for one metric will be null for some dimension combinations. Because the + operator returns null when either input is null, the entire result becomes null for those dimensions. Additionally, queries that aggregate a single metric can filter out null values more efficiently.

Applied, thanks!

unresolving this in reference to #134373 (comment)

(the code font didn't get applied as shown in this claude+me suggestion -- you can "edit" the comment to copy the markdown instead of plain text)

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

# Conflicts: # docs/reference/query-languages/esql/_snippets/functions/description/rate.md # docs/reference/query-languages/esql/kibana/definition/functions/rate.json # docs/reference/query-languages/esql/kibana/docs/functions/rate.md # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/Idelta.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/Increase.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/Irate.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/Rate.java

Co-authored-by: Marci W <[email protected]>

docs/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/AvgOverTime.java

leemthompo

Sorry I forgot to group my latest comments in a review :/

I think this is looking very good now and big thanks for all the iteration @kkrik-es. Once those final fixes are through, I'd consider this ready to merge from a writing perspective. It might be good to a final pair of technical eyeballs on this, because it's information rich!

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

Co-authored-by: Liam Thompson <[email protected]>

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

…ts.md Co-authored-by: Liam Thompson <[email protected]>

kkrik-es · 2025-09-25T11:15:28Z

Thanks @leemthompo and @marciw for the review and the suggestions! @felixbarny mind taking another look, since this is much expanded? You can also check out the structure in the doc preview.

felixbarny

Just a couple of nits

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

felixbarny · 2025-09-25T13:55:44Z

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

+These functions are implicitly evaluated per time series, then aggregated by group using a secondary aggregation
+function. For example:


Nit: maybe improve the wording a bit on this.

Suggested change

These functions are implicitly evaluated per time series, then aggregated by group using a secondary aggregation

function. For example:

These functions are implicitly evaluated per time series.

A secondary secondary aggregation function, in combination with a `BY` grouping expression, allows to group by a set of dimensions.

For example:

I'll stick with Marci's version.

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

docs/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md

felixbarny · 2025-09-25T14:03:07Z

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

+such as `SUM()`, as outer aggregation functions. Using a time series aggregation
+in combination with an inner function causes an error. For example, the


Suggested change

such as `SUM()`, as outer aggregation functions. Using a time series aggregation

in combination with an inner function causes an error. For example, the

such as `SUM()`, as outer aggregation functions.

Nesting time series aggregation functions causes an error.

For example, the

Hm nesting is not very clear? Using them as inner functions can still be called "nesting".

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

yannis-roussos

Thank you @kkrik-es, stellar work on those docs! I have only a minor comment for your consideration.

Additional minor notes on updates not included in this PR:

Nitpick: On the time-series aggregation functions, why did we go with bucket on the examples instead of TBUCKET?
Minor: On the time-series aggregation functions some of the descriptions are very laconic. We could maybe use some of the descriptions of the regular aggregation functions or link to them.
A special case that I would propose to expand its description is the rate command as it is so critical for time series.

Approving on my side as everything mentioned on my comments is minor or optional.

yannis-roussos · 2025-09-25T15:53:40Z

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md

+This query calculates the total rate of search requests (tracked by the `search_requests` counter) per host and hour. The `RATE()` 
+function is applied per time series in hourly buckets. These rates are summed for each
+host and hourly bucket (since each host can map to multiple time series).


Optional suggestion for your consideration: I think that an example would benefit introducing this concept that may be new to some users (either here on on the examples section bellow)

Something that would showcase what happens on each step, using a diagram or tables with real dimension values. We do so for various important commands on the intro sections for ES|QL, so why not also here? I am thinking about this cause this is the most important concept, and then everything else is built on top of it.

Also, should we highlight a little bit more the special place of TBUCKET being used on both the inner and outer aggregations? We mention it but not state it explicitly

I think it's better to expand this in a blog post. I do think we should have more examples below to cover more use cases, without going into too many details of how things work.

Sounds like a good plan, thank you Kostas!

leemthompo

Thanks @kkrik-es!

Co-authored-by: Felix Barnsteiner <[email protected]>

kkrik-es · 2025-09-26T11:31:12Z

@yannis-roussos I plan to follow up on expanding the documentation for rate, as it's special. The rest are more straightforward, but can be expanded as needed.

Also, documentation for the time-series functions was added before we had the TBUCKET command.. Can be refined separately.

kkrik-es · 2025-09-26T11:35:04Z

Thanks all for the thorough reviews and recommendations! I'll keep iterating, will send some extra PRs.

Add documentation for TS source

6463013

kkrik-es self-assigned this Sep 9, 2025

kkrik-es added >docs General docs changes Team:StorageEngine :StorageEngine/ES|QL Timeseries / metrics / logsdb capabilities in ES|QL labels Sep 9, 2025

elasticsearchmachine added the v9.2.0 label Sep 9, 2025

github-actions bot deployed to docs-preview September 9, 2025 15:25 View deployment

add toc ref

8677ce0

github-actions bot deployed to docs-preview September 9, 2025 15:46 View deployment

update nav

49630b7

github-actions bot deployed to docs-preview September 9, 2025 15:52 View deployment

update

3beebc4

github-actions bot deployed to docs-preview September 9, 2025 18:15 View deployment

kkrik-es requested review from dnhatn, felixbarny, leemthompo and marciw September 10, 2025 11:31

kkrik-es mentioned this pull request Sep 10, 2025

[ES|QL] Add a TS variation of GenerativeIT #133804

Merged

Merge branch 'main' into tsdb/ts-doc

4104e1a

github-actions bot deployed to docs-preview September 10, 2025 15:07 View deployment

dnhatn approved these changes Sep 10, 2025

View reviewed changes

marciw reviewed Sep 10, 2025

View reviewed changes

Apply suggestions from code review

c860567

Co-authored-by: Marci W <[email protected]>

github-actions bot had a problem deploying to docs-preview September 11, 2025 08:36 Failure

Update ts.md

5d23ffe

github-actions bot had a problem deploying to docs-preview September 11, 2025 08:37 Failure

Merge branch 'main' into tsdb/ts-doc

4ec4daf

github-actions bot deployed to docs-preview September 11, 2025 08:39 View deployment

remove note

4d7e593

marciw reviewed Sep 25, 2025

View reviewed changes

kkrik-es and others added 3 commits September 25, 2025 10:34

Apply suggestions from code review

ee1df8b

Co-authored-by: Marci W <[email protected]>

Update ts.md

9276b86

leemthompo reviewed Sep 25, 2025

View reviewed changes

docs/reference/query-languages/esql/functions-operators/time-series-aggregation-functions.md Show resolved Hide resolved

leemthompo reviewed Sep 25, 2025

View reviewed changes

...ql/src/main/java/org/elasticsearch/xpack/esql/expression/function/aggregate/AvgOverTime.java Outdated Show resolved Hide resolved

leemthompo reviewed Sep 25, 2025

View reviewed changes

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md Outdated Show resolved Hide resolved

kkrik-es and others added 2 commits September 25, 2025 13:51

Apply suggestions from code review

b0600a9

Co-authored-by: Liam Thompson <[email protected]>

Apply suggestions from code review

aa587e8

Co-authored-by: Liam Thompson <[email protected]>

leemthompo reviewed Sep 25, 2025

View reviewed changes

docs/reference/query-languages/esql/_snippets/commands/layout/ts.md Outdated Show resolved Hide resolved

kkrik-es and others added 2 commits September 25, 2025 14:09

update function description

1f17af3

Update docs/reference/query-languages/esql/_snippets/commands/layout/…

71fa400

…ts.md Co-authored-by: Liam Thompson <[email protected]>

Merge branch 'main' into tsdb/ts-doc

013a5e0

felixbarny approved these changes Sep 25, 2025

View reviewed changes

yannis-roussos approved these changes Sep 25, 2025

View reviewed changes

leemthompo approved these changes Sep 25, 2025

View reviewed changes

marciw approved these changes Sep 25, 2025

View reviewed changes

kkrik-es and others added 3 commits September 26, 2025 14:22

Apply suggestions from code review

2b1832c

Co-authored-by: Felix Barnsteiner <[email protected]>

Merge branch 'main' into tsdb/ts-doc

59541ae

fix from review

a2706ea

Update ts.md

2921961

kkrik-es mentioned this pull request Sep 26, 2025

Adding docs-producing tests for delta, idelta, increase, irate #135490

Merged

kkrik-es merged commit b659c3a into elastic:main Sep 26, 2025
34 checks passed

kkrik-es deleted the tsdb/ts-doc branch September 26, 2025 12:47

kkrik-es mentioned this pull request Sep 26, 2025

[Internal]: Document time-series querying in ES|QL elastic/docs-content#2852

Closed

		These functions are implicitly evaluated per time series, then aggregated by group using a secondary aggregation
		function. For example:

		such as `SUM()`, as outer aggregation functions. Using a time series aggregation
		in combination with an inner function causes an error. For example, the

Add documentation for TS source #134373

Add documentation for TS source #134373

Uh oh!

Conversation

kkrik-es commented Sep 9, 2025

Uh oh!

github-actions bot commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

github-actions bot commented Sep 9, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

Uh oh!

dnhatn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marciw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kkrik-es commented Sep 24, 2025

Uh oh!

marciw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marciw Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

leemthompo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kkrik-es commented Sep 25, 2025

Uh oh!

felixbarny left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Sep 9, 2025 •

edited

Loading

marciw Sep 25, 2025 •

edited

Loading

kkrik-es commented Sep 26, 2025 •

edited

Loading