diff --git a/src/assets/images/log-explorer/expressions.png b/src/assets/images/log-explorer/expressions.png new file mode 100644 index 00000000000000..0e0d3febf350a0 Binary files /dev/null and b/src/assets/images/log-explorer/expressions.png differ diff --git a/src/assets/images/log-explorer/not.png b/src/assets/images/log-explorer/not.png new file mode 100644 index 00000000000000..9148ace016f1ab Binary files /dev/null and b/src/assets/images/log-explorer/not.png differ diff --git a/src/assets/images/log-explorer/scalar-aggregate-functions.png b/src/assets/images/log-explorer/scalar-aggregate-functions.png new file mode 100644 index 00000000000000..3c783d552d8d04 Binary files /dev/null and b/src/assets/images/log-explorer/scalar-aggregate-functions.png differ diff --git a/src/assets/images/log-explorer/supported-sql-grammar-graph.png b/src/assets/images/log-explorer/supported-sql-grammar-graph.png index 490442533dd89a..f33bacb5672951 100644 Binary files a/src/assets/images/log-explorer/supported-sql-grammar-graph.png and b/src/assets/images/log-explorer/supported-sql-grammar-graph.png differ diff --git a/src/content/docs/log-explorer/custom-dashboards.mdx b/src/content/docs/log-explorer/custom-dashboards.mdx index db9bf6e032a06c..4e488ab8e6d604 100644 --- a/src/content/docs/log-explorer/custom-dashboards.mdx +++ b/src/content/docs/log-explorer/custom-dashboards.mdx @@ -2,7 +2,7 @@ pcx_content_type: reference title: Custom dashboards sidebar: - order: 3 + order: 4 --- Custom dashboards allow you to create tailored dashboards to monitor application security, performance, and usage. You can create monitors for ongoing monitoring of a previous incident, use them to identify indicators of suspicious activity, and access templates to help you get started. diff --git a/src/content/docs/log-explorer/log-search.mdx b/src/content/docs/log-explorer/log-search.mdx index a8f55cdf3885d5..0918103df71141 100644 --- a/src/content/docs/log-explorer/log-search.mdx +++ b/src/content/docs/log-explorer/log-search.mdx @@ -9,49 +9,6 @@ import { TabItem, Tabs, Render } from "~/components"; Log Explorer enables you to store and explore your Cloudflare logs directly within the Cloudflare dashboard or API, giving you visibility into your logs without the need to forward them to third-party services. Logs are stored on Cloudflare's global network using the R2 object storage platform and can be queried via the dashboard or SQL API. -## SQL queries supported - -The diagram below displays the example sql grammar for `SELECT` statements as a railroad syntax diagram: - -![Supported SQL grammar](~/assets/images/log-explorer/supported-sql-grammar-graph.png) - -Any path from left to right forms a valid query. There is a limit of 25 predicates in the `WHERE` clause. Predicates can be grouped using parenthesis. If the `LIMIT` clause is not specified, then the default limit of 10,000 is applied. The maximum number for the `LIMIT` clause is 10,000. Results are returned in descending order by time. - -Examples of queries include: - -- `SELECT * FROM table WHERE (a = 1 OR b = "hello") AND c < 25.89` -- `SELECT a, b, c FROM table WHERE d >= "GB" LIMIT 10` - -### SELECT - -The `SELECT` clause specifies the columns that you want to retrieve from the database tables. It can include individual column names, expressions, or even wildcard characters to select all columns. - -### FROM - -The `FROM` clause specifies the tables from which to retrieve data. It indicates the source of the data for the `SELECT` statement. - -### WHERE - -The `WHERE` clause filters the rows returned by a query based on specified conditions. It allows you to specify conditions that must be met for a row to be included in the result set. - -### GROUP BY - -The `GROUP BY` clause is used to group rows that have the same values into summary rows. - -### ORDER BY - -The `ORDER BY` clause is used to sort the result set by one or more columns in ascending or descending order. - -### LIMIT - -The `LIMIT` clause is used to constrain the number of rows returned by a query. It is often used in conjunction with the `ORDER BY` clause to retrieve the top `N` rows or to implement pagination. - -:::note - -Log Explorer does not support `JOIN`, `DDL`, `DML`, or `EXPLAIN` queries. - -::: - ## Use Log Explorer You can filter and view your logs via the Cloudflare dashboard or the API. diff --git a/src/content/docs/log-explorer/sql-queries.mdx b/src/content/docs/log-explorer/sql-queries.mdx new file mode 100644 index 00000000000000..a7b6601ab9a1cb --- /dev/null +++ b/src/content/docs/log-explorer/sql-queries.mdx @@ -0,0 +1,170 @@ +--- +pcx_content_type: concept +title: SQL queries supported +sidebar: + order: 3 +--- + +import { Details } from "~/components" + +This page outlines the SQL features supported by Log Explorer, including common aggregation functions, expressions, and query clauses. + +The diagram below illustrates the general shape of a valid query supported in Log Explorer. It shows how standard SQL clauses — such as `SELECT`, `WHERE`, `GROUP BY`, and `ORDER BY` — can be composed to form supported queries. + +![Supported SQL grammar](~/assets/images/log-explorer/supported-sql-grammar-graph.png) + +Examples of queries include: + +- `SELECT * FROM table WHERE (a = 1 OR b = "hello") AND c < 25.89` +- `SELECT a, b, c FROM table WHERE d >= "GB" LIMIT 10` + +:::note +- A default `LIMIT` of 10,000 is applied if the `LIMIT` clause is omitted. +- The `WHERE` clause supports up to 25 predicates, which can be grouped using parentheses. +::: + +### SQL Clauses in detail + +The following SQL clauses define the structure and logic of queries in Log Explorer: + +- `SELECT` - The `SELECT` clause specifies the columns that you want to retrieve from the database tables. It can include individual column names, expressions, or even wildcard characters to select all columns. +- `FROM` - The `FROM` clause specifies the tables from which to retrieve data. It indicates the source of the data for the `SELECT` statement. +- `WHERE` - The `WHERE` clause filters the rows returned by a query based on specified conditions. It allows you to specify conditions that must be met for a row to be included in the result set. +- `SELECT DISTINCT` - Removes duplicate rows from the result set. +- `GROUP BY` - Groups rows for aggregation. The `GROUP BY` clause is used to group rows that have the same values into summary rows. +- `ORDER BY` - Sorts the result set. The `ORDER BY` clause is used to sort the result set by one or more columns in ascending or descending order. +- `LIMIT` - Restricts the number of rows returned. The `LIMIT` clause is used to constrain the number of rows returned by a query. It is often used in conjunction with the `ORDER BY` clause to retrieve the top `N` rows or to implement pagination. +- `OFFSET` - Skips a specified number of rows before returning results. + +The sections that follow break down the remaining components shown in the diagram — such as aggregation functions, string and numeric expressions, and supported operators — in more detail. + +## Functions + +Log Explorer supports a range of SQL functions to transform, evaluate, or summarize data. These include scalar and aggregation functions. + +### Scalar functions + +These help manipulate or evaluate values (often strings): + +- `ARRAY_CONTAINS(array, element)` – Checks if the array contains the element. + +
+ `ARRAY_CONTAINS(['US', 'CA'], ClientCountry)` + + Returns rows where `ClientCountry` is either `US` or `CA`. +
+ +- `SUBSTRING(string, from_number, for_number)` – Extracts part of a string. + +
+ `SUBSTRING(ClientRequestPath, 0, 5)` + + Extracts the first `5` characters from `ClientRequestPath`. +
+ +- `LOWER(string)` – Converts to lowercase. + +
+ `LOWER(ClientRequestUserAgent)` + + Converts the user agent string to lowercase. +
+ +- `UPPER(string)` – Converts to uppercase. + +
+ `UPPER(ClientCountry)` + + Converts the country code to uppercase. +
+ +### Aggregation functions + +Used to perform calculations on sets of rows: + +- `SUM(expression)` – Total of values. + +
+ `SUM(ClientRequestBytes)` + + Adds up the total number of bytes requested by clients. +
+ +- `MIN(expression)` – Minimum value. + +
+ `MIN(OriginResponseDurationMs)` + + Finds the shortest response time from origin servers. +
+ +- `MAX(expression)` – Maximum value. + +
+ `MAX(OriginResponseDurationMs)` + + Finds the longest response time. +
+ +- `COUNT(expression)` – Number of rows (can be all rows or non-null values). + +
+ `COUNT(ClientRequestUserAgent)` + + Counts how many rows have a user agent value. +
+ +- `COUNT(DISTINCT expression)` – Number of distinct non-null values. + +
+ `COUNT(DISTINCT ClientIP)` + + Counts how many unique client IPs made requests. +
+ +- `AVG(expression)` – Average of numeric values. + +
+ `AVG(OriginResponseDurationMs)` + + Computes the average origin response time in milliseconds. +
+ +The diagram below represents the grammar for SQL expressions including scalar and aggregate functions. + +![Scalar and aggregate functions](~/assets/images/log-explorer/scalar-aggregate-functions.png) + +## Expressions + +Conditions or logic used in queries: + +- `CASE WHEN` – Conditional logic (like if-else). +- `AS` – Alias for columns or tables. +- `LIKE` – Pattern matching. +- `IN (list)` – Checks if a value is in a list. +- `BETWEEN ... AND ...` – Checks if a value is within a range. +- `Unary operator` – Operates on one operand (for example, `-5`). +- `Binary operator` – Operates on two operands (for example, `5 + 3`). +- `Nested Expressions` – Expression wrapped with parentheses, like `( x > y )` or `( 1 )`. +- `Compound identifier` – Multi-part name (for example, `schema.table.column`). +- `Array` – A collection of values (supported differently across SQL dialects). +- `Literals` - represent values such as strings, numbers, or arrays. + +The diagram below represents the grammar for SQL expressions, detailing the various forms an expression can take, including columns, literals, functions, operators, and aliases. + +![SQL expressions](~/assets/images/log-explorer/expressions.png) + +The diagram below defines the grammar for unary operators, which operate on a single operand (for example, negation or logical `NOT`): + +![Grammar for unary operators](~/assets/images/log-explorer/not.png) + +## Binary Operators + +Used for arithmetic, comparison, logical operations: + +- Arithmetic: `+`, `-`, `*`, `/`, `%` (modulo) +- Comparison: `>`, `<`, `>=`, `<=`, `=`, `!=` (or `<>`)` +- Logical: `AND`, `OR`, `XOR` +- Bitwise: `&`, `|`, `^`, `>>`, `<<` +- String concat: `||` +