Skip to content

Commit f5083ed

Browse files
luigidellaquilaluigidellaquila
committed
Add docs for ES|QL Query Log
1 parent 9eeea2e commit f5083ed

File tree

2 files changed

+138
-0
lines changed

2 files changed

+138
-0
lines changed

‎explore-analyze/query-filter/languages/esql-query-log.md‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
---
2+
applies_to:
3+
stack: ga
4+
serverless: ga
5+
navigation_title: "Query log"
6+
mapped_pages:
7+
- https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-query-log.html
8+
---
9+
10+
11+
12+
# ES|QL query log [esql-query-log]
13+
14+
15+
Query Log allows to log ES|QL queries based on their execution time.
16+
You can use these logs to investigate, analyze or troubleshoot your cluster’s historical ES|QL performance.
17+
18+
ES|QL query log reports task duration at coordinator level, but might not encompass the full task execution time observed on the client. For example, logs don’t surface HTTP network delays.
19+
20+
Events that meet the specified threshold are emitted into [{{es}} logging](../../../deploy-manage/monitor/logging-configuration/update-elasticsearch-logging-levels.md).
21+
22+
These logs can be found in local {{es}} service logs directory. Slow log files have a suffix of `_esql_querylog.json`.
23+
24+
## Query log format [query-log-format]
25+
26+
The following is an example of a successful query event in the query log:
27+
28+
```js
29+
{
30+
"@timestamp": "2025-03-11T08:39:50.076Z",
31+
"log.level": "TRACE",
32+
"auth.type": "REALM",
33+
"elasticsearch.querylog.planning.took": 3108666,
34+
"elasticsearch.querylog.planning.took_millis": 3,
35+
"elasticsearch.querylog.query": "from index | limit 100",
36+
"elasticsearch.querylog.search_type": "ESQL",
37+
"elasticsearch.querylog.success": true,
38+
"elasticsearch.querylog.took": 8050416,
39+
"elasticsearch.querylog.took_millis": 8,
40+
"user.name": "elastic-admin",
41+
"user.realm": "default_file",
42+
"ecs.version": "1.2.0",
43+
"service.name": "ES_ECS",
44+
"event.dataset": "elasticsearch.esql_querylog",
45+
"process.thread.name": "elasticsearch[runTask-0][esql_worker][T#12]",
46+
"log.logger": "esql.querylog.query",
47+
"elasticsearch.cluster.uuid": "KZo1V7TcQM-O6fnqMm1t_g",
48+
"elasticsearch.node.id": "uPgRE2TrSfa9IvnUpNT1Uw",
49+
"elasticsearch.node.name": "runTask-0",
50+
"elasticsearch.cluster.name": "runTask"
51+
}
52+
```
53+
54+
The following is an example of a failing query event in the query log:
55+
56+
```js
57+
{
58+
"@timestamp": "2025-03-11T08:41:54.172Z",
59+
"log.level": "TRACE",
60+
"auth.type": "REALM",
61+
"elasticsearch.querylog.error.message": "line 1:15: mismatched input 'limitxyz' expecting {DEV_CHANGE_POINT, 'enrich', 'dissect', 'eval', 'grok', 'limit', 'sort', 'stats', 'where', DEV_INLINESTATS, DEV_FORK, 'lookup', DEV_JOIN_LEFT, DEV_JOIN_RIGHT, DEV_LOOKUP, 'mv_expand', 'drop', 'keep', DEV_INSIST, 'rename'}",
62+
"elasticsearch.querylog.error.type": "org.elasticsearch.xpack.esql.parser.ParsingException",
63+
"elasticsearch.querylog.query": "from person | limitxyz 100",
64+
"elasticsearch.querylog.search_type": "ESQL",
65+
"elasticsearch.querylog.success": false,
66+
"elasticsearch.querylog.took": 963750,
67+
"elasticsearch.querylog.took_millis": 0,
68+
"user.name": "elastic-admin",
69+
"user.realm": "default_file",
70+
"ecs.version": "1.2.0",
71+
"service.name": "ES_ECS",
72+
"event.dataset": "elasticsearch.esql_querylog",
73+
"process.thread.name": "elasticsearch[runTask-0][search][T#16]",
74+
"log.logger": "esql.querylog.query",
75+
"elasticsearch.cluster.uuid": "KZo1V7TcQM-O6fnqMm1t_g",
76+
"elasticsearch.node.id": "uPgRE2TrSfa9IvnUpNT1Uw",
77+
"elasticsearch.node.name": "runTask-0",
78+
"elasticsearch.cluster.name": "runTask"
79+
}
80+
```
81+
82+
83+
## Enable query logging [enable-query-log]
84+
85+
You can enable query logging at cluster level.
86+
87+
By default, all thresholds are set to `-1`, which results in no events being logged.
88+
89+
Query log thresholds can be enabled for the four logging levels: `trace`, `debug`, `info`, and `warn`.
90+
91+
To view the current query log settings, use the [get cluster settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings):
92+
93+
```console
94+
GET _cluster/settings?filter_path=*.esql.querylog.*
95+
```
96+
97+
You can use the `esql.querylog.include.user` setting to append `user.*` and `auth.type` fields to slow log entries. These fields contain information about the user who triggered the request.
98+
99+
The following snippet adjusts all available ES|QL query log settings [update cluster settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings):
100+
101+
```console
102+
PUT /_cluster/settings
103+
{
104+
"transient": {
105+
"esql.querylog.threshold.warn": "10s",
106+
"esql.querylog.threshold.info": "5s",
107+
"esql.querylog.threshold.debug": "2s",
108+
"esql.querylog.threshold.trace": "500ms",
109+
"esql.querylog.include.user": true
110+
}
111+
}
112+
```
113+
114+
115+
116+
## Best practices for query logging [troubleshoot-query-log]
117+
118+
Logging slow requests can be resource intensive to your {{es}} cluster depending on the qualifying traffic’s volume. For example, emitted logs might increase the index disk usage of your [{{es}} monitoring](../../../deploy-manage/monitor/stack-monitoring.md) cluster. To reduce the impact of slow logs, consider the following:
119+
120+
* Set high thresholds to reduce the number of logged events.
121+
* Enable slow logs only when troubleshooting.
122+
123+
If you aren’t sure how to start investigating traffic issues, consider enabling the `warn` threshold with a high `30s` threshold at the index level using the [update cluster settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings):
124+
125+
* Enable for search requests:
126+
127+
```console
128+
PUT /_cluster/settings
129+
{
130+
"transient": {
131+
"esql.querylog.include.user": true,
132+
"esql.querylog.threshold.warn": "30s",
133+
}
134+
}
135+
```
136+
137+

‎explore-analyze/toc.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ toc:
2020
- file: query-filter/languages/esql-cross-clusters.md
2121
- file: query-filter/languages/esql-examples.md
2222
- file: query-filter/languages/esql-task-management.md
23+
- file: query-filter/languages/esql-query-log.md
2324
- file: query-filter/languages/sql.md
2425
children:
2526
- file: query-filter/languages/sql-overview.md

0 commit comments

Comments
 (0)