Skip to content

Commit ba8222b

Browse files
[Log explorer] Adds new page for SQL queries (#24005)
* Adds new page for SQL queries * Apply suggestions from code review Co-authored-by: marciocloudflare <[email protected]> * Adds details componet to examples * Final review --------- Co-authored-by: marciocloudflare <[email protected]>
1 parent 32f67d4 commit ba8222b

File tree

7 files changed

+171
-44
lines changed

7 files changed

+171
-44
lines changed
202 KB
Loading
24.3 KB
Loading
36.4 KB
Loading
133 KB
Loading

src/content/docs/log-explorer/custom-dashboards.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
pcx_content_type: reference
33
title: Custom dashboards
44
sidebar:
5-
order: 3
5+
order: 4
66
---
77

88
Custom dashboards allow you to create tailored dashboards to monitor application security, performance, and usage. You can create monitors for ongoing monitoring of a previous incident, use them to identify indicators of suspicious activity, and access templates to help you get started.

src/content/docs/log-explorer/log-search.mdx

Lines changed: 0 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -9,49 +9,6 @@ import { TabItem, Tabs, Render } from "~/components";
99

1010
Log Explorer enables you to store and explore your Cloudflare logs directly within the Cloudflare dashboard or API, giving you visibility into your logs without the need to forward them to third-party services. Logs are stored on Cloudflare's global network using the R2 object storage platform and can be queried via the dashboard or SQL API.
1111

12-
## SQL queries supported
13-
14-
The diagram below displays the example sql grammar for `SELECT` statements as a railroad syntax diagram:
15-
16-
![Supported SQL grammar](~/assets/images/log-explorer/supported-sql-grammar-graph.png)
17-
18-
Any path from left to right forms a valid query. There is a limit of 25 predicates in the `WHERE` clause. Predicates can be grouped using parenthesis. If the `LIMIT` clause is not specified, then the default limit of 10,000 is applied. The maximum number for the `LIMIT` clause is 10,000. Results are returned in descending order by time.
19-
20-
Examples of queries include:
21-
22-
- `SELECT * FROM table WHERE (a = 1 OR b = "hello") AND c < 25.89`
23-
- `SELECT a, b, c FROM table WHERE d >= "GB" LIMIT 10`
24-
25-
### SELECT
26-
27-
The `SELECT` clause specifies the columns that you want to retrieve from the database tables. It can include individual column names, expressions, or even wildcard characters to select all columns.
28-
29-
### FROM
30-
31-
The `FROM` clause specifies the tables from which to retrieve data. It indicates the source of the data for the `SELECT` statement.
32-
33-
### WHERE
34-
35-
The `WHERE` clause filters the rows returned by a query based on specified conditions. It allows you to specify conditions that must be met for a row to be included in the result set.
36-
37-
### GROUP BY
38-
39-
The `GROUP BY` clause is used to group rows that have the same values into summary rows.
40-
41-
### ORDER BY
42-
43-
The `ORDER BY` clause is used to sort the result set by one or more columns in ascending or descending order.
44-
45-
### LIMIT
46-
47-
The `LIMIT` clause is used to constrain the number of rows returned by a query. It is often used in conjunction with the `ORDER BY` clause to retrieve the top `N` rows or to implement pagination.
48-
49-
:::note
50-
51-
Log Explorer does not support `JOIN`, `DDL`, `DML`, or `EXPLAIN` queries.
52-
53-
:::
54-
5512
## Use Log Explorer
5613

5714
You can filter and view your logs via the Cloudflare dashboard or the API.
Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
---
2+
pcx_content_type: concept
3+
title: SQL queries supported
4+
sidebar:
5+
order: 3
6+
---
7+
8+
import { Details } from "~/components"
9+
10+
This page outlines the SQL features supported by Log Explorer, including common aggregation functions, expressions, and query clauses.
11+
12+
The diagram below illustrates the general shape of a valid query supported in Log Explorer. It shows how standard SQL clauses — such as `SELECT`, `WHERE`, `GROUP BY`, and `ORDER BY` — can be composed to form supported queries.
13+
14+
![Supported SQL grammar](~/assets/images/log-explorer/supported-sql-grammar-graph.png)
15+
16+
Examples of queries include:
17+
18+
- `SELECT * FROM table WHERE (a = 1 OR b = "hello") AND c < 25.89`
19+
- `SELECT a, b, c FROM table WHERE d >= "GB" LIMIT 10`
20+
21+
:::note
22+
- A default `LIMIT` of 10,000 is applied if the `LIMIT` clause is omitted.
23+
- The `WHERE` clause supports up to 25 predicates, which can be grouped using parentheses.
24+
:::
25+
26+
### SQL Clauses in detail
27+
28+
The following SQL clauses define the structure and logic of queries in Log Explorer:
29+
30+
- `SELECT` - The `SELECT` clause specifies the columns that you want to retrieve from the database tables. It can include individual column names, expressions, or even wildcard characters to select all columns.
31+
- `FROM` - The `FROM` clause specifies the tables from which to retrieve data. It indicates the source of the data for the `SELECT` statement.
32+
- `WHERE` - The `WHERE` clause filters the rows returned by a query based on specified conditions. It allows you to specify conditions that must be met for a row to be included in the result set.
33+
- `SELECT DISTINCT` - Removes duplicate rows from the result set.
34+
- `GROUP BY` - Groups rows for aggregation. The `GROUP BY` clause is used to group rows that have the same values into summary rows.
35+
- `ORDER BY` - Sorts the result set. The `ORDER BY` clause is used to sort the result set by one or more columns in ascending or descending order.
36+
- `LIMIT` - Restricts the number of rows returned. The `LIMIT` clause is used to constrain the number of rows returned by a query. It is often used in conjunction with the `ORDER BY` clause to retrieve the top `N` rows or to implement pagination.
37+
- `OFFSET` - Skips a specified number of rows before returning results.
38+
39+
The sections that follow break down the remaining components shown in the diagram — such as aggregation functions, string and numeric expressions, and supported operators — in more detail.
40+
41+
## Functions
42+
43+
Log Explorer supports a range of SQL functions to transform, evaluate, or summarize data. These include scalar and aggregation functions.
44+
45+
### Scalar functions
46+
47+
These help manipulate or evaluate values (often strings):
48+
49+
- `ARRAY_CONTAINS(array, element)` – Checks if the array contains the element.
50+
51+
<Details header="Example">
52+
`ARRAY_CONTAINS(['US', 'CA'], ClientCountry)`
53+
54+
Returns rows where `ClientCountry` is either `US` or `CA`.
55+
</Details>
56+
57+
- `SUBSTRING(string, from_number, for_number)` – Extracts part of a string.
58+
59+
<Details header="Example">
60+
`SUBSTRING(ClientRequestPath, 0, 5)`
61+
62+
Extracts the first `5` characters from `ClientRequestPath`.
63+
</Details>
64+
65+
- `LOWER(string)` – Converts to lowercase.
66+
67+
<Details header="Example">
68+
`LOWER(ClientRequestUserAgent)`
69+
70+
Converts the user agent string to lowercase.
71+
</Details>
72+
73+
- `UPPER(string)` – Converts to uppercase.
74+
75+
<Details header="Example">
76+
`UPPER(ClientCountry)`
77+
78+
Converts the country code to uppercase.
79+
</Details>
80+
81+
### Aggregation functions
82+
83+
Used to perform calculations on sets of rows:
84+
85+
- `SUM(expression)` – Total of values.
86+
87+
<Details header="Example">
88+
`SUM(ClientRequestBytes)`
89+
90+
Adds up the total number of bytes requested by clients.
91+
</Details>
92+
93+
- `MIN(expression)` – Minimum value.
94+
95+
<Details header="Example">
96+
`MIN(OriginResponseDurationMs)`
97+
98+
Finds the shortest response time from origin servers.
99+
</Details>
100+
101+
- `MAX(expression)` – Maximum value.
102+
103+
<Details header="Example">
104+
`MAX(OriginResponseDurationMs)`
105+
106+
Finds the longest response time.
107+
</Details>
108+
109+
- `COUNT(expression)` – Number of rows (can be all rows or non-null values).
110+
111+
<Details header="Example">
112+
`COUNT(ClientRequestUserAgent)`
113+
114+
Counts how many rows have a user agent value.
115+
</Details>
116+
117+
- `COUNT(DISTINCT expression)` – Number of distinct non-null values.
118+
119+
<Details header="Example">
120+
`COUNT(DISTINCT ClientIP)`
121+
122+
Counts how many unique client IPs made requests.
123+
</Details>
124+
125+
- `AVG(expression)` – Average of numeric values.
126+
127+
<Details header="Example">
128+
`AVG(OriginResponseDurationMs)`
129+
130+
Computes the average origin response time in milliseconds.
131+
</Details>
132+
133+
The diagram below represents the grammar for SQL expressions including scalar and aggregate functions.
134+
135+
![Scalar and aggregate functions](~/assets/images/log-explorer/scalar-aggregate-functions.png)
136+
137+
## Expressions
138+
139+
Conditions or logic used in queries:
140+
141+
- `CASE WHEN` – Conditional logic (like if-else).
142+
- `AS` – Alias for columns or tables.
143+
- `LIKE` – Pattern matching.
144+
- `IN (list)` – Checks if a value is in a list.
145+
- `BETWEEN ... AND ...` – Checks if a value is within a range.
146+
- `Unary operator` – Operates on one operand (for example, `-5`).
147+
- `Binary operator` – Operates on two operands (for example, `5 + 3`).
148+
- `Nested Expressions` – Expression wrapped with parentheses, like `( x > y )` or `( 1 )`.
149+
- `Compound identifier` – Multi-part name (for example, `schema.table.column`).
150+
- `Array` – A collection of values (supported differently across SQL dialects).
151+
- `Literals` - represent values such as strings, numbers, or arrays.
152+
153+
The diagram below represents the grammar for SQL expressions, detailing the various forms an expression can take, including columns, literals, functions, operators, and aliases.
154+
155+
![SQL expressions](~/assets/images/log-explorer/expressions.png)
156+
157+
The diagram below defines the grammar for unary operators, which operate on a single operand (for example, negation or logical `NOT`):
158+
159+
![Grammar for unary operators](~/assets/images/log-explorer/not.png)
160+
161+
## Binary Operators
162+
163+
Used for arithmetic, comparison, logical operations:
164+
165+
- Arithmetic: `+`, `-`, `*`, `/`, `%` (modulo)
166+
- Comparison: `>`, `<`, `>=`, `<=`, `=`, `!=` (or `<>`)`
167+
- Logical: `AND`, `OR`, `XOR`
168+
- Bitwise: `&`, `|`, `^`, `>>`, `<<`
169+
- String concat: `||`
170+

0 commit comments

Comments
 (0)