Skip to content

Commit 3c07043

Browse files
Updates
1 parent d8c1a11 commit 3c07043

File tree

6 files changed

+143
-76
lines changed

6 files changed

+143
-76
lines changed
50.5 KB
Loading

src/content/docs/log-explorer/api.mdx

Lines changed: 68 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,72 @@
1-
---
2-
pcx_content_type: reference
3-
title: Log Explorer API
4-
sidebar:
5-
order: 5
6-
---
1+
Configuration and Log searches are also available via a public API.
2+
## Authentication
3+
4+
Log Explorer is available to users with the following permissions:
5+
6+
- **Logs Edit**: users with Logs Edit permissions can enable datasets.
7+
- **Logs Read**: users with Logs Read permissions can run queries via the UI or API.
8+
9+
Note that these permissions exist at the account and zone level and you need the appropriate permission level for the datasets you wish to query.
710

8-
Log searches are available via the [API](https://developers.cloudflare.com/log-explorer/).
11+
Authentication with the API can be done via an authentication header or API token. Append your API call with either of the following additional parameters.
12+
13+
- **Authentication header**
914

10-
Log Explorer exposes a query endpoint that uses familiar SQL syntax for querying your logs generated with Cloudflare's network.
15+
- `X-Auth-Email` - the Cloudflare account email address associated with the domain
16+
- `X-Auth-Key` - the Cloudflare API key
17+
18+
- **API token**
19+
20+
- `Authorization: Bearer <API_TOKEN>` To create an appropriately scoped API token, refer to [Create API token](/fundamentals/api/get-started/create-token/) documentation. Copy and paste the token into the authorization parameter for your API call.
21+
22+
## Manage Datasets
23+
24+
Use the Log Explorer API to enable Log Explorer for each dataset you wish to store. It may take a few minutes after a log stream is enabled before you can view the logs.
25+
26+
The following curl command is an example for enabling the zone-level dataset `http_requests`, as well as the expected response when the command succeeds.
27+
28+
```bash
29+
curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logs/explorer/datasets \
30+
--header "Authorization: Bearer <API_TOKEN>" \
31+
--header "Content-Type: application/json" \
32+
--data '{
33+
"dataset": "http_requests"
34+
}'
35+
```
36+
37+
```json
38+
{
39+
"result": {
40+
"dataset": "http_requests",
41+
"object_type": "zone",
42+
"object_id": "<ZONE ID>",
43+
"created_at": "2025-06-03T14:33:16Z",
44+
"updated_at": "2025-06-03T14:33:16Z",
45+
"dataset_id": "01973635f7e273a1964a02f4d4502499",
46+
"enabled": true
47+
},
48+
"success": true,
49+
"errors": [],
50+
"messages": []
51+
}
52+
```
1153

12-
For example, to find an HTTP request with a specific [Ray ID](/fundamentals/reference/cloudflare-ray-id/), you can perform the following SQL query:
54+
If you would like to enable an account-level dataset, replace `zones/{zone_id}` with `accounts/{account_id}` in the curl command. For example:
55+
56+
```bash
57+
curl https://api.cloudflare.com/client/v4/accounts/{account_id}/logs/explorer/datasets \
58+
--header "Authorization: Bearer <API_TOKEN>" \
59+
--header "Content-Type: application/json" \
60+
--data '{
61+
"dataset": "access_requests"
62+
}'
63+
```
64+
65+
## Query data
66+
67+
Log Explorer includes a SQL API for submitting queries.
68+
69+
For example, to find an HTTP request with a specific [Ray ID](/fundamentals/reference/cloudflare-ray-id/), and use the following SQL query:
1370

1471
```bash
1572
curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logs/explorer/query/sql \
@@ -36,7 +93,7 @@ Which returns the following HTTP request details:
3693
}
3794
```
3895

39-
A example to find Cloudflare Access requests with selected columns from a specific timeframe, you can perform the following SQL query:
96+
Another example to find Cloudflare Access requests with selected columns from a specific timeframe, you can perform the following SQL query:
4097

4198
```bash
4299
curl https://api.cloudflare.com/client/v4/account/{account_id}/logs/explorer/query/sql \
@@ -66,24 +123,4 @@ Which returns the following request details:
66123
"errors": [],
67124
"messages": []
68125
}
69-
```
70-
71-
## Authentication
72-
73-
Log Search is available to users with the following permissions:
74-
75-
- **Logs Edit**: users with Logs Edit permissions can enable datasets.
76-
- **Logs Read**: users with Logs Read permissions can run queries via the UI or API.
77-
78-
Note that these permissions exist at the account and zone level and you need the appropriate permission level for the datasets you wish to query.
79-
80-
Authentication with the API can be done via an authentication header or API token. Append your API call with either of the following additional parameters.
81-
82-
- **Authentication header**
83-
84-
- `X-Auth-Email` - the Cloudflare account email address associated with the domain
85-
- `X-Auth-Key` - the Cloudflare API key
86-
87-
- **API token**
88-
89-
- `Authorization: Bearer <API_TOKEN>` To create an appropriately scoped API token, refer to [Create API token](/fundamentals/api/get-started/create-token/) documentation. Copy and paste the token into the authorization parameter for your API call.
126+
```

src/content/docs/log-explorer/index.mdx

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,20 @@ import { Description, Feature, RelatedProduct } from "~/components"
1111
Store and explore your Cloudflare logs directly within the Cloudflare dashboard or API.
1212
</Description>
1313

14-
Log Explorer is a log storage platform that allows security teams and developers to analyze, investigate, and monitor attacks natively within the Cloudflare dashboard. Log Explorer is built on [Cloudflare R2](/r2/) which allows you to search your logs using SQL queries.
1514

16-
With Log Explorer, you can monitor security and performance issues with custom dashboards, investigate and troubleshoot issues with Log Search, save time and collaborate with saved queries.
15+
Log Explorer is Cloudflare's native observability and forensics product that enables security teams and developers to analyze, investigate, and monitor issues directly from the Cloudflare dashboard, without the expense and complexity of forwarding logs to third party tools.
16+
17+
Log Explorer provides access to Cloudflare logs with all the context available within the Cloudflare platform. You can monitor security and performance issues with custom dashboards or investigate and troubleshoot issues with log search. Benefits include:
18+
19+
- **Reduced cost and complexity**: Drastically reduce the expense and operational overhead associated with forwarding, storing, and analyzing terabytes of log data in external tools.
20+
- **Faster detection and triage**: Access Cloudflare-native logs directly, eliminating cumbersome data pipelines and the ingest lags that delay critical security insights.
21+
- **Accelerated investigations with full context**: Investigate incidents with Cloudflare's unparalleled contextual data, accelerating your analysis and understanding of "What exactly happened?" and "How did it happen?"
22+
- **Minimal recovery time**: Seamlessly transition from investigation to action with direct mitigation capabilities via the Cloudflare platform.
1723

1824
## Features
1925

2026
<Feature header="Log Search" href="/log-explorer/log-search/">
21-
Search logs enable you to store and explore your Cloudflare logs directly within the Cloudflare dashboard or API.
27+
Search logs enable you to store and explore your Cloudflare logs directly within the Cloudflare dashboard or [API](/log-explorer/api/).
2228
</Feature>
2329

2430
<Feature header="Custom dashboards" href="/log-explorer/custom-dashboards/">
@@ -36,7 +42,7 @@ Manage configuration and perform queries via the API.
3642
## Related products
3743

3844
<RelatedProduct header="Logpush" href="/logs/" product="logs">
39-
Detailed logs that contain metadata generated by Cloudflare products helpful for debugging, identifying configuration adjustments, and creating analytics.
45+
Forward Cloudflare logs to third party tools for debugging, identifying configuration adjustments, and creating analytics.
4046
</RelatedProduct>
4147

4248
<RelatedProduct header="Analytics" href="/analytics/" product="analytics">

src/content/docs/log-explorer/log-search.mdx

Lines changed: 56 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,48 @@ import { TabItem, Tabs, Render } from "~/components";
99

1010
Log Explorer enables you to store and explore your Cloudflare logs directly within the Cloudflare dashboard or API. Giving you visibility into your logs without the need to forward them to third parties. Logs are stored on Cloudflare's global network using the R2 object storage platform and can be queried via the Dashboard or SQL API.
1111

12-
## Supported datasets
12+
## SQL queries supported
13+
14+
The diagram below displays the example sql grammar for SELECT statements as a railroad syntax diagram:
15+
16+
![Supported sql grammar](~/assets/images/log-explorer/supported-sql-grammar-graph.png)
17+
18+
Any path from left to right forms a valid query. There is a limit of 25 predicates in the `WHERE` clause. Predicates can be grouped using parenthesis. If the `LIMIT` clause is not specified, then the default limit of 10000 is applied. The maximum number for the `LIMIT` clause is 10000. Results are returned in descending order by time.
19+
20+
Examples of queries include:
21+
22+
- `SELECT * FROM table WHERE (a = 1 OR b = "hello") AND c < 25.89`
23+
- `SELECT a, b, c FROM table WHERE d >= "GB" LIMIT 10`
24+
25+
### SELECT
26+
27+
The `SELECT` clause specifies the columns that you want to retrieve from the database tables. It can include individual column names, expressions, or even wildcard characters to select all columns.
28+
29+
### FROM
30+
31+
The `FROM` clause specifies the tables from which to retrieve data. It indicates the source of the data for the `SELECT` statement.
32+
33+
### WHERE
34+
35+
The `WHERE` clause filters the rows returned by a query based on specified conditions. It allows you to specify conditions that must be met for a row to be included in the result set.
36+
37+
### GROUP BY
38+
39+
The `GROUP BY` clause is used to group rows that have the same values into summary rows.
40+
41+
### ORDER BY
1342

14-
Log Explorer currently supports:
43+
The `ORDER BY` clause is used to sort the result set by one or more columns in ascending or descending order.
44+
45+
### LIMIT
1546

16-
- [HTTP requests](/logs/reference/log-fields/zone/http_requests/) (`FROM http_requests`)
17-
- [Firewall events](/logs/reference/log-fields/zone/firewall_events/) (`FROM firewall_events`)
47+
The `LIMIT` clause is used to constrain the number of rows returned by a query. It is often used in conjunction with the `ORDER BY` clause to retrieve the top N rows or to implement pagination.
48+
49+
:::note
50+
51+
Log Explorer does not support `JOINs`, `DDL`, `DML`, or `EXPLAIN` queries.
52+
53+
:::
1854

1955
## Use Log Explorer
2056

@@ -30,8 +66,9 @@ You can filter and view your logs via the Cloudflare dashboard or the API.
3066
8. Select **Run query** when you are done. The results are displayed below within the **Query results** section.
3167

3268
:::note
33-
You can also access the Log Explorer dashboard directly from the [Security Analytics dashboard](/waf/analytics/security-analytics/#logs). When doing so, the filters you applied in Security Analytics will automatically carry over to your query in Log Explorer.
34-
:::
69+
70+
71+
3572

3673
For example, to find an HTTP request with a specific [Ray ID](/fundamentals/reference/cloudflare-ray-id/), go to **Custom SQL**, and enter the following SQL query:
3774

@@ -47,6 +84,7 @@ For example, to find an HTTP request with a specific [Ray ID](/fundamentals/refe
4784
LIMIT 1
4885
```
4986

87+
5088
Another example to find Cloudflare Access requests with selected columns from a specific timeframe, you can perform the following SQL query:
5189

5290
```sql
@@ -65,10 +103,22 @@ Another example to find Cloudflare Access requests with selected columns from a
65103
WHERE Date >= '2025-02-06' AND Date <= '2025-02-06' AND CreatedAt >= '2025-02-06T12:28:39Z' AND CreatedAt <= '2025-02-06T12:58:39Z'
66104
```
67105

106+
107+
108+
109+
110+
111+
68112
### Save queries
69113

70114
After selecting all the fields for your query, you can save it by selecting **Save query**. Provide a name and description to help identify it later. To view your saved and recent queries, select **Queries** — they will appear in a side panel where you can insert a new query, or delete any query.
71115

116+
## Integrated with Security Analytics
117+
118+
You can also access the Log Explorer dashboard directly from the [Security Analytics dashboard](/waf/analytics/security-analytics/#logs). When doing so, the filters you applied in Security Analytics will automatically carry over to your query in Log Explorer.
119+
120+
:::
121+
72122
## Optimize your queries
73123

74124
All the tables supported by Log Explorer contain a special column called `date`, which helps to narrow down the amount of data that is scanned to respond to your query, resulting in faster query response times. The value of `date` must be in the form of `YYYY-MM-DD`. For example, to query logs that occurred on October 12, 2023, add the following to your `WHERE` clause: `date = '2023-10-12'`. The column supports the standard operators of `<`, `>`, and `=`.
@@ -102,40 +152,6 @@ All the tables supported by Log Explorer contain a special column called `date`,
102152
- Omit `ORDER BY` and `LIMIT` clauses. These clauses can slow down queries, especially when dealing with large datasets. For queries that return a large number of records, reduce the time frame instead of limiting to the newest `N` records from a broader time frame.
103153
- Select only necessary columns. For example, replace `SELECT *` with the list of specific columns you need. You can also use `SELECT RayId` as a first iteration and follow up with a query that filters by the Ray IDs to retrieve additional columns. Additionally, you can use `SELECT COUNT(*)` to probe for time frames with matching records without retrieving the full dataset.
104154

105-
## SQL queries supported
106-
107-
These are the SQL query clauses supported by Log Explorer.
108-
109-
### SELECT
110-
111-
The `SELECT` clause specifies the columns that you want to retrieve from the database tables. It can include individual column names, expressions, or even wildcard characters to select all columns.
112-
113-
### FROM
114-
115-
The `FROM` clause specifies the tables from which to retrieve data. It indicates the source of the data for the `SELECT` statement.
116-
117-
### WHERE
118-
119-
The `WHERE` clause filters the rows returned by a query based on specified conditions. It allows you to specify conditions that must be met for a row to be included in the result set.
120-
121-
### GROUP BY
122-
123-
The `GROUP BY` clause is used to group rows that have the same values into summary rows.
124-
125-
### ORDER BY
126-
127-
The `ORDER BY` clause is used to sort the result set by one or more columns in ascending or descending order.
128-
129-
### LIMIT
130-
131-
The `LIMIT` clause is used to constrain the number of rows returned by a query. It is often used in conjunction with the `ORDER BY` clause to retrieve the top N rows or to implement pagination.
132-
133-
:::note
134-
135-
Log Explorer does not support `JOINs`, `DDL`, `DML`, or `EXPLAIN` queries.
136-
137-
:::
138-
139155
## FAQs
140156

141157
### Which fields (or columns) are available for querying?

src/content/docs/log-explorer/manage-datasets.mdx

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,14 @@ import { TabItem, Tabs, Render } from "~/components";
99

1010
Log Explorer allows you to enable or disable which datasets are available to query in Log Search.
1111

12+
## Supported datasets
13+
14+
Log Explorer currently supports:
15+
16+
- [HTTP requests](/logs/reference/log-fields/zone/http_requests/) (`FROM http_requests`)
17+
- [Firewall events](/logs/reference/log-fields/zone/firewall_events/) (`FROM firewall_events`)
18+
19+
1220
## Enable Log Explorer
1321

1422
In order for Log Explorer to begin storing logs, you need to enable the desired datasets. You can do this via the dashboard or the API.

src/content/docs/logs/log-explorer.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,7 @@ You can choose the output format with an HTTP `Accept` header, as shown in the t
207207
| CSV | `text/csv` | Yes |
208208
| Plain text | `text/plain` | Yes |
209209

210-
## Optimizing your queries
210+
## Optimize your queries
211211

212212
All the tables supported by Log Explorer contain a special column called `date`, which helps to narrow down the amount of data that is scanned to respond to your query, resulting in faster query response times. The value of `date` must be in the form of `YYYY-MM-DD`. For example, to query logs that occurred on October 12, 2023, add the following to your `WHERE` clause: `date = '2023-10-12'`. The column supports the standard operators of `<`, `>`, and `=`.
213213

0 commit comments

Comments
 (0)