Skip to content

Commit ea3ab5b

Browse files
committed
[workers-observability] query builder docs
1 parent fffa3a3 commit ea3ab5b

File tree

9 files changed

+184
-2
lines changed

9 files changed

+184
-2
lines changed
148 KB
Loading
229 KB
Loading
239 KB
Loading
275 KB
Loading
276 KB
Loading
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
pcx_content_type: navigation
3+
title: Investigate
4+
sidebar:
5+
order: 3
6+
group:
7+
hideIndex: true
8+
---
9+
10+
import { Badge, Stream } from "~/components";
11+
12+
Explore all your Workers telemetry data from the Cloudflare dashboard.
13+
14+
## [Query Builder](/workers/observability/investigate/query-builder) <Badge text="New" variant="tip" size="large" />
15+
16+
Write structured queries to investigate and visualize your telemetry data.
Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
---
2+
pcx_content_type: concept
3+
title: Query Builder
4+
head: []
5+
description: Write structured queries to investigate and visualize your telemetry data.
6+
sidebar:
7+
order: 1
8+
badge:
9+
variant: tip
10+
text: New
11+
12+
---
13+
14+
import { TabItem, Tabs, Steps, Render, WranglerConfig, YouTube } from "~/components"
15+
16+
The Query Builder helps you write structured queries to investigate and visualize your telemtry data. The Query Builder searches the Workers Observability dataset, which currently includes all logs stored by [Workers Logs](/workers/observability/logs/workers-logs).
17+
18+
The Query Builder can be found in the [Workers' Observability tab in the Cloudflare dashboard](https://dash.cloudflare.com/?to=/:account/workers-and-pages/observability/investigate).
19+
20+
<YouTube id="nu4pTU8fR78" />
21+
22+
## Enable Query Builder
23+
24+
The Query Builder is available to all developers and requires no enablement. Queries search all Workers Logs stored by Cloudflare. If you have not yet enabled Workers Logs, you can do so by adding the following setting to your [Worker's Wrangler file](/workers/observability/logs/workers-logs/#enable-workers-logs) and redeploying your Worker.
25+
26+
<WranglerConfig>
27+
```toml
28+
[observability]
29+
enabled = true
30+
31+
[observability.logs]
32+
invocation_logs = true
33+
head_sampling_rate = 1 # optional. default = 1.
34+
```
35+
</WranglerConfig>
36+
37+
## Write a Query in the Cloudflare Dashboard
38+
39+
<Steps>
40+
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/?to=/:account/workers/services/view/:worker/observability/investigate/) and select your account.
41+
2. In Account Home, go to **Workers & Pages**.
42+
3. Select **Observability** in the left-hand navigation panel, and then the **Investigate** tab.
43+
4. Select a **Visualization**.
44+
5. Optional: Add fields to Filter, Group By, Order By, and Limit. For more information, see what [composes a query](/workers/observability/investigate/query-builder/query-composition).
45+
6. Optional: Select the appropriate time range.
46+
7. Click **Run**. The query will automatically run whenever changes are made.
47+
</Steps>
48+
49+
## Query Composition
50+
51+
### Visualization
52+
53+
The Query Builder supports many visualization operators, including:
54+
- Count
55+
- Count Distinct
56+
- Min, Max, Sum, Average
57+
- Standard Deviation, Variance
58+
- P001, P01, P05, P10, P25, Median (P50), P75, P90, P95, P99, P999
59+
60+
You can add multiple visualizations in a single query. Each visualization renders a graph. A single summary table is also returned, which shows the raw query results.
61+
62+
![Example of showing the Query Builder with multiple visualization](~/assets/images/workers-observability/query-builder-visualization.png)
63+
64+
All methods are aggregate functions. Most methods operate on a specific field in the log event. `Count` is an exception, and is an aggregate function that returns the number of log events matching the filter conditions.
65+
66+
### Filter
67+
68+
Filters help return the columns that match the specified conditions. Filters have three components: a key, an operator, and a value.
69+
70+
The key is any field in a log event. For example, you may choose `$workers.cpuTimeMs` or `$metadata.message`.
71+
72+
The operator is a logical condition that evaluates to true or false. For numeric values, valid conditions are `Equals`, `Does not equal`, Greater`, `Greater or equals`, `Less`, and `Less or equals`. For string values, valid conditions are `Equals`, `Does not equal`, `Includes`, `Does not include`, `Regex`, `Exists`, `Does not exist`, and `Starts with`.
73+
74+
The value for a numeric field is an integer. The value for a string field is any string.
75+
76+
To add a filter:
77+
78+
<Steps>
79+
1. Click **+** in the **Filter** section.
80+
2. Select **Select key...** and input a key name. For example, `$workers.cpuTimeMs`.
81+
3. Select the operator and change it to the operator best suited. For example, `Greater than`.
82+
4. Select **Select value...** and input a value. For example, `100`.
83+
</Steps>
84+
85+
When you run the query with the filter specified above, only log events where `$workers.cpuTimeMs > 100` will be returned.
86+
87+
Adding multiple filters combines them with an AND operator, meaning that only events matching all the filters will be returned.
88+
89+
### Search
90+
91+
Search is a text filter that returns only events containing the specified text. Search can be helpful as a quick filtering mechanism, or to search for unique indentifiable values in your logs.
92+
93+
### Group By
94+
95+
Group By combines rows that have the same value into summary rows. For example, if a query adds `$workers.event.request.cf.country` as a Group By field, then the summary table will group by country.
96+
97+
### Order By
98+
99+
Order By affects how the results are sorted in the summary table. If `asc` is selected, the results are sorted in ascending order -- from least to greatest. If `desc` is selected, the results are sorted in descending order -- from greatest to least.
100+
101+
### Limit
102+
103+
Limit restricts the number of results returned. When paired with [Order By](/workers/observability/investigate/#order-by), it can be used to return the "top" or "first" N results.
104+
105+
### Select Time Range
106+
107+
When you select a time range, you specify the time interval where you want to look for matching events. The retention period is dependent on your [plan type](/workers/observability/logs/workers-logs/#pricing).
108+
109+
## Viewing Query Results
110+
111+
There are three views for queries: Visualizations, Invocations, and Events.
112+
113+
### Visualizations tab
114+
115+
The Visualizations tab shows graphs and a summary table for the query.
116+
117+
### Invocations tab
118+
119+
The Invocation tab shows all logs, grouped by by the invocation, and ordered by timestamp. Only invocations matching the query criteria are returned.
120+
121+
### Events tab
122+
123+
The Events tab shows all logs, ordered by timestamp. Only events matching the query criteria are returned. The Events tab can be customized to add additional fields in the view.
124+
125+
## Save queries
126+
127+
It is recommended to save queries that may be reused for future investigations. You can save a query with a name, description, and custom tags by selecting the 'Save Query' button. Queries are saved at the account-level and are accessible to all users in the account.
128+
129+
Saved queries can be re-run by selecting the relevant query from the 'Queries' tab. You can edit the query and save edits.
130+
131+
Queries can be starred by users. Starred queries are unique to the user, and not to the account.
132+
133+
## Delete queries
134+
135+
Saved queries can be deleted from the 'Queries' tab. If you delete a query, the query is deleted for all users in the account.
136+
137+
<Steps>
138+
1. Select the [Queries](https://dash.cloudflare.com/?to=/:account/workers-and-pages/observability/queries) tab in the Observability dashboard.
139+
2. On the right-hand side, select the three dots for additional actions.
140+
3. Select 'Delete Query' and follow the instructions.
141+
</Steps>
142+
143+
## Share queries
144+
145+
Saved queries are assigned a unique URL and can be shared with any user in the account.
146+
147+
## Example
148+
149+
Let's construct a query to find and debug all paths that respond with 500 errors. First, let's create a base query. In this base query, we want to visualize by
150+
the raw event count. We can add a filter for `$workers.event.response.status` that is greater than 300 so we can identify all paths that may be experiencing
151+
unexpected behavior. Then, we can group by `$workers.event.request.path` and `$workers.event.response.status` to identify the number of requests that were
152+
affected by this behavior.
153+
154+
![Constructing a query](~/assets/images/workers-observability/query-builder-ex1-query.png)
155+
156+
The results show that the `/actuator/env` path has been experiencing 500s. Let us apply a filter for this path and investigate.
157+
158+
![Adding an additional field to the query](~/assets/images/workers-observability/query-builder-ex1-query-with-filter.png)
159+
160+
Now, let's investigate by clicking on the 'Invocations' tab. We can see that there were two logged invocations of this error.
161+
162+
![Examining the Invocations tab in the Query Builder](~/assets/images/workers-observability/query-builder-ex1-invocations.png)
163+
164+
We can expand a single invocation to view the relevant logs, and continue to debug.
165+
166+
![Viewing the logs for a single Invocation](~/assets/images/workers-observability/query-builder-ex1-invocation-logs.png)

src/content/docs/workers/observability/logs/index.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ import { Badge, Stream } from "~/components";
1111

1212
Logs are an important component of a developer's toolkit to troubleshoot and diagnose application issues and maintaining system health. The Cloudflare Developer Platform offers many tools to help developers manage their application's logs.
1313

14-
## [Workers Logs](/workers/observability/logs/workers-logs) <Badge text="New" variant="tip" size="large" />
14+
## [Workers Logs](/workers/observability/logs/workers-logs)
1515

1616
Automatically ingest, filter, and analyze logs emitted from Cloudflare Workers in the Cloudflare dashboard.
1717

src/content/docs/workers/observability/logs/workers-logs.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,7 @@ head_sampling_rate = 0.01 # 1% sampling rate
193193
| Maximum logs per account per day<sup>1</sup> | 5 Billion |
194194
| Maximum log size<sup>2</sup> | 256 KB |
195195

196-
<sup>1</sup> While Workers Logs is in open beta, there is a daily limit of 5 billion logs per account per day. After the limit is exceed, a 1% head-based sample will be applied for the remainder of the day.
196+
<sup>1</sup> There is a daily limit of 5 billion logs per account per day. After the limit is exceed, a 1% head-based sample will be applied for the remainder of the day.
197197

198198
<sup>2</sup> A single log has a maximum size limit of [256 KB](/workers/platform/limits/#log-size). Logs exceeding that size will be truncated and the log's `$cloudflare.truncated` field will be set to true.
199199

0 commit comments

Comments
 (0)