|
| 1 | +--- |
| 2 | +pcx_content_type: concept |
| 3 | +title: Query Builder |
| 4 | +head: [] |
| 5 | +description: Write structured queries to investigate and visualize your telemetry data. |
| 6 | +sidebar: |
| 7 | + order: 1 |
| 8 | + badge: |
| 9 | + variant: tip |
| 10 | + text: New |
| 11 | + |
| 12 | +--- |
| 13 | + |
| 14 | +import { TabItem, Tabs, Steps, Render, WranglerConfig, YouTube, Markdown } from "~/components" |
| 15 | + |
| 16 | +The Query Builder helps you write structured queries to investigate and visualize your telemtry data. The Query Builder searches the Workers Observability dataset, which currently includes all logs stored by [Workers Logs](/workers/observability/logs/workers-logs). |
| 17 | + |
| 18 | +The Query Builder can be found in the [Workers' Observability tab in the Cloudflare dashboard](https://dash.cloudflare.com/?to=/:account/workers-and-pages/observability/investigate). |
| 19 | + |
| 20 | +<YouTube id="nu4pTU8fR78" /> |
| 21 | + |
| 22 | +## Enable Query Builder |
| 23 | + |
| 24 | +The Query Builder is available to all developers and requires no enablement. Queries search all Workers Logs stored by Cloudflare. If you have not yet enabled Workers Logs, you can do so by adding the following setting to your [Worker's Wrangler file](/workers/observability/logs/workers-logs/#enable-workers-logs) and redeploying your Worker. |
| 25 | + |
| 26 | +<WranglerConfig> |
| 27 | + ```toml |
| 28 | + [observability] |
| 29 | + enabled = true |
| 30 | + |
| 31 | + [observability.logs] |
| 32 | + invocation_logs = true |
| 33 | + head_sampling_rate = 1 # optional. default = 1. |
| 34 | + ``` |
| 35 | +</WranglerConfig> |
| 36 | + |
| 37 | +## Write a Query in the Cloudflare Dashboard |
| 38 | + |
| 39 | +<Steps> |
| 40 | +1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/?to=/:account/workers/services/view/:worker/observability/investigate/) and select your account. |
| 41 | +2. In Account Home, go to **Workers & Pages**. |
| 42 | +3. Select **Observability** in the left-hand navigation panel, and then the **Investigate** tab. |
| 43 | +4. Select a **Visualization**. |
| 44 | +5. Optional: Add fields to Filter, Group By, Order By, and Limit. For more information, see what [composes a query](/workers/observability/investigate/query-builder/query-composition). |
| 45 | +6. Optional: Select the appropriate time range. |
| 46 | +7. Click **Run**. The query will automatically run whenever changes are made. |
| 47 | +</Steps> |
| 48 | + |
| 49 | +## Query Composition |
| 50 | + |
| 51 | +### Visualization |
| 52 | + |
| 53 | +The Query Builder supports many visualization operators, including: |
| 54 | + |
| 55 | +| Function | Arguments | Description | |
| 56 | +| --- | --- | --- | |
| 57 | +| **Count** | n/a | The total number of rows matching the query conditions | |
| 58 | +| **Count Distinct** | any field | The number of occurrences of the unique values in the dataset | |
| 59 | +| **Min** | numeric field | The smallest value for the field in the dataset | |
| 60 | +| **Max** | numeric field | The largest value for the field in the dataset | |
| 61 | +| **Sum** | numeric field | The total of all of the values for the field in the dataset | |
| 62 | +| **Average** | numeric field | The average of the field in the dataset | |
| 63 | +| **Standard Deviation** | numeric field | The standard deviation of the field in the dataset | |
| 64 | +| **Variance** | numeric field | The variance of the field in the dataset | |
| 65 | +| **P001** | numeric field | The value of the field below which 0.1% of the data falls | |
| 66 | +| **P01** | numeric field | The value of the field below with 1% of the data falls | |
| 67 | +| **P05** | numeric field | The value of the field below with 5% of the data falls | |
| 68 | +| **P10** | numeric field | The value of the field below with 10% of the data falls | |
| 69 | +| **P25** | numeric field | The value of the field below with 25% of the data falls | |
| 70 | +| **Median (P50)** | numeric field | The value of the field below with 50% of the data falls | |
| 71 | +| **P75** | numeric field | The value of the field below with 75% of the data falls | |
| 72 | +| **P90** | numeric field | The value of the field below with 90% of the data falls | |
| 73 | +| **P95** | numeric field | The value of the field below with 95% of the data falls | |
| 74 | +| **P99** | numeric field | The value of the field below with 99% of the data falls | |
| 75 | +| **P999** | numeric field | The value of the field below with 99.9% of the data falls | |
| 76 | + |
| 77 | +You can add multiple visualizations in a single query. Each visualization renders a graph. A single summary table is also returned, which shows the raw query results. |
| 78 | + |
| 79 | + |
| 80 | + |
| 81 | +All methods are aggregate functions. Most methods operate on a specific field in the log event. `Count` is an exception, and is an aggregate function that returns the number of log events matching the filter conditions. |
| 82 | + |
| 83 | +### Filter |
| 84 | + |
| 85 | +Filters help return the columns that match the specified conditions. Filters have three components: a key, an operator, and a value. |
| 86 | + |
| 87 | +The key is any field in a log event. For example, you may choose `$workers.cpuTimeMs` or `$metadata.message`. |
| 88 | + |
| 89 | +The operator is a logical condition that evaluates to true or false. See the table below for supported conditions: |
| 90 | +| Data Type | Valid Conditions (Operators) | |
| 91 | +| --- | --- | |
| 92 | +| Numeric | Equals, Does not equal, Greater, Greater or equals, Less, Less or equals, Exists, Does not exist | |
| 93 | +| String | Equals, Does not equal, Includes, Does not include, Regex, Exists, Does not exist, Starts with | |
| 94 | + |
| 95 | +The value for a numeric field is an integer. The value for a string field is any string. |
| 96 | + |
| 97 | +To add a filter: |
| 98 | + |
| 99 | +<Steps> |
| 100 | +1. Click **+** in the **Filter** section. |
| 101 | +2. Select **Select key...** and input a key name. For example, `$workers.cpuTimeMs`. |
| 102 | +3. Select the operator and change it to the operator best suited. For example, `Greater than`. |
| 103 | +4. Select **Select value...** and input a value. For example, `100`. |
| 104 | +</Steps> |
| 105 | + |
| 106 | +When you run the query with the filter specified above, only log events where `$workers.cpuTimeMs > 100` will be returned. |
| 107 | + |
| 108 | +Adding multiple filters combines them with an AND operator, meaning that only events matching all the filters will be returned. |
| 109 | + |
| 110 | +### Search |
| 111 | + |
| 112 | +Search is a text filter that returns only events containing the specified text. Search can be helpful as a quick filtering mechanism, or to search for unique indentifiable values in your logs. |
| 113 | + |
| 114 | +### Group By |
| 115 | + |
| 116 | +Group By combines rows that have the same value into summary rows. For example, if a query adds `$workers.event.request.cf.country` as a Group By field, then the summary table will group by country. |
| 117 | + |
| 118 | +### Order By |
| 119 | + |
| 120 | +Order By affects how the results are sorted in the summary table. If `asc` is selected, the results are sorted in ascending order -- from least to greatest. If `desc` is selected, the results are sorted in descending order -- from greatest to least. |
| 121 | + |
| 122 | +### Limit |
| 123 | + |
| 124 | +Limit restricts the number of results returned. When paired with [Order By](/workers/observability/investigate/#order-by), it can be used to return the "top" or "first" N results. |
| 125 | + |
| 126 | +### Select Time Range |
| 127 | + |
| 128 | +When you select a time range, you specify the time interval where you want to look for matching events. The retention period is dependent on your [plan type](/workers/observability/logs/workers-logs/#pricing). |
| 129 | + |
| 130 | +## Viewing Query Results |
| 131 | + |
| 132 | +There are three views for queries: Visualizations, Invocations, and Events. |
| 133 | + |
| 134 | +### Visualizations tab |
| 135 | + |
| 136 | +The Visualizations tab shows graphs and a summary table for the query. |
| 137 | + |
| 138 | + |
| 139 | + |
| 140 | +### Invocations tab |
| 141 | + |
| 142 | +The Invocation tab shows all logs, grouped by by the invocation, and ordered by timestamp. Only invocations matching the query criteria are returned. |
| 143 | + |
| 144 | +p |
| 145 | + |
| 146 | + |
| 147 | +### Events tab |
| 148 | + |
| 149 | +The Events tab shows all logs, ordered by timestamp. Only events matching the query criteria are returned. The Events tab can be customized to add additional fields in the view. |
| 150 | + |
| 151 | + |
| 152 | + |
| 153 | +## Save queries |
| 154 | + |
| 155 | +It is recommended to save queries that may be reused for future investigations. You can save a query with a name, description, and custom tags by selecting the 'Save Query' button. Queries are saved at the account-level and are accessible to all users in the account. |
| 156 | + |
| 157 | +Saved queries can be re-run by selecting the relevant query from the 'Queries' tab. You can edit the query and save edits. |
| 158 | + |
| 159 | +Queries can be starred by users. Starred queries are unique to the user, and not to the account. |
| 160 | + |
| 161 | +## Delete queries |
| 162 | + |
| 163 | +Saved queries can be deleted from the 'Queries' tab. If you delete a query, the query is deleted for all users in the account. |
| 164 | + |
| 165 | +<Steps> |
| 166 | +1. Select the [Queries](https://dash.cloudflare.com/?to=/:account/workers-and-pages/observability/queries) tab in the Observability dashboard. |
| 167 | +2. On the right-hand side, select the three dots for additional actions. |
| 168 | +3. Select 'Delete Query' and follow the instructions. |
| 169 | +</Steps> |
| 170 | + |
| 171 | +## Share queries |
| 172 | + |
| 173 | +Saved queries are assigned a unique URL and can be shared with any user in the account. |
| 174 | + |
| 175 | +## Example |
| 176 | + |
| 177 | +Let's construct a query to find and debug all paths that respond with 500 errors. First, let's create a base query. In this base query, we want to visualize by |
| 178 | +the raw event count. We can add a filter for `$workers.event.response.status` that is greater than 300 so we can identify all paths that may be experiencing |
| 179 | +unexpected behavior. Then, we can group by `$workers.event.request.path` and `$workers.event.response.status` to identify the number of requests that were |
| 180 | +affected by this behavior. |
| 181 | + |
| 182 | + |
| 183 | + |
| 184 | +The results show that the `/actuator/env` path has been experiencing 500s. Let us apply a filter for this path and investigate. |
| 185 | + |
| 186 | + |
| 187 | + |
| 188 | +Now, let's investigate by clicking on the 'Invocations' tab. We can see that there were two logged invocations of this error. |
| 189 | + |
| 190 | + |
| 191 | + |
| 192 | +We can expand a single invocation to view the relevant logs, and continue to debug. |
| 193 | + |
| 194 | + |
0 commit comments