Skip to content

Commit fccf9a9

Browse files
Maddy-Cloudflarepedrosousa
authored and
sdnts
committed
[Log Explorer] Log explorer docs (cloudflare#23049)
* [Log Explorer] Log explorer docs * Adding redirects * Fixing codeowner file * Updating intro * Deleting sections and making updates * Adding manage datasets and api page + updates * Updates * add title * Apply suggestions from code review Co-authored-by: Pedro Sousa <[email protected]> * Addressing Pedro suggestions --------- Co-authored-by: Pedro Sousa <[email protected]>
1 parent 23106a0 commit fccf9a9

File tree

11 files changed

+490
-1
lines changed

11 files changed

+490
-1
lines changed

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@
9090
/src/content/docs/registrar/ @dcpena @cloudflare/pcx-technical-writing
9191
/src/content/docs/rules/ @pedrosousa @cloudflare/pcx-technical-writing
9292
/src/content/docs/ruleset-engine/ @pedrosousa @cloudflare/pcx-technical-writing
93+
/src/content/docs/log-explorer/ @angelampcosta @cloudflare/pcx-technical-writing
9394

9495
# Developer Platform
9596

‎public/__redirects‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,7 @@
188188
/analytics/graphql-api/tutorials/build-your-own-analytics/ /analytics/graphql-api/tutorials/ 301
189189
/analytics/graphql-api/tutorials/export-graphql-to-csv/ /analytics/graphql-api/tutorials/ 301
190190
/analytics/analytics-integrations/google-cloud/ /analytics/analytics-integrations/ 301
191+
/analytics/dashboards/ /log-explorer/custom-dashboards/ 301
191192

192193
# email-security
193194
/email-security/reporting/search/detection-search/ /email-security/reporting/search/ 301
@@ -933,6 +934,7 @@
933934
/logs/reference/logpush-api-configuration/filters/ /logs/reference/filters/ 301
934935
# Non-slashed version is being used in the Cloudflare dashboard
935936
/logs/reference/logpush-api-configuration/examples/example-logpush-curl/ /logs/tutorials/examples/example-logpush-curl/ 301
937+
/logs/log-explorer/ /log-explorer/log-search/ 301
936938

937939
# magic-firewall
938940
/magic-firewall/reference/examples/ /magic-firewall/how-to/add-rules/ 301

‎src/assets/images/log-explorer/supported-sql-grammar-graph.png‎

50.5 KB
Loading

‎src/content/docs/log-explorer/api.mdx‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
pcx_content_type: reference
3+
title: Log Explorer API
4+
sidebar:
5+
order: 5
6+
---
7+
8+
Configuration and Log searches are also available via a public API.
9+
10+
## Authentication
11+
12+
Log Explorer is available to users with the following permissions:
13+
14+
- **Logs Edit**: users with Logs Edit permissions can enable datasets.
15+
- **Logs Read**: users with Logs Read permissions can run queries via the UI or API.
16+
17+
Note that these permissions exist at the account and zone level and you need the appropriate permission level for the datasets you wish to query.
18+
19+
Authentication with the API can be done via an API token or API key with an email. Refer to [Create API token](/fundamentals/api/get-started/create-token/) for further instructions.
20+
21+
## Query data
22+
23+
Log Explorer includes a SQL API for submitting queries.
24+
25+
For example, to find an HTTP request with a specific [Ray ID](/fundamentals/reference/cloudflare-ray-id/), use the following SQL query:
26+
27+
```bash
28+
curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logs/explorer/query/sql \
29+
--header "Authorization: Bearer <API_TOKEN>" \
30+
--url-query query="SELECT clientRequestScheme, clientRequestHost, clientRequestMethod, edgeResponseStatus, clientRequestUserAgent FROM http_requests WHERE RayID = '806c30a3cec56817' LIMIT 1"
31+
```
32+
33+
This command returns the following HTTP request details:
34+
35+
```json
36+
{
37+
"result": [
38+
{
39+
"clientrequestscheme": "https",
40+
"clientrequesthost": "example.com",
41+
"clientrequestmethod": "GET",
42+
"clientrequestuseragent": "curl/7.88.1",
43+
"edgeresponsestatus": 200
44+
}
45+
],
46+
"success": true,
47+
"errors": [],
48+
"messages": []
49+
}
50+
```
51+
52+
As another example, you could find Cloudflare Access requests with selected columns from a specific timeframe by performing the following SQL query:
53+
54+
```bash
55+
curl https://api.cloudflare.com/client/v4/account/{account_id}/logs/explorer/query/sql \
56+
--header "Authorization: Bearer <API_TOKEN>" \
57+
--url-query query="SELECT CreatedAt, AppDomain, AppUUID, Action, Allowed, Country, RayID, Email, IPAddress, UserUID FROM access_requests WHERE Date >= '2025-02-06' AND Date <= '2025-02-06' AND CreatedAt >= '2025-02-06T12:28:39Z' AND CreatedAt <= '2025-02-06T12:58:39Z'"
58+
```
59+
60+
This command returns the following request details:
61+
62+
```json
63+
{
64+
"result": [
65+
{
66+
"createdat": "2025-01-14T18:17:55Z",
67+
"appdomain": "example.com",
68+
"appuuid": "a66b4ab0-ccdf-4d60-a6d0-54a59a827d92",
69+
"action": "login",
70+
"allowed": true,
71+
"country": "us",
72+
"rayid": "90fbb07c0b316957",
73+
"email": "[email protected]",
74+
"ipaddress": "1.2.3.4",
75+
"useruid": "52859e81-711e-4de0-8b31-283336060e79"
76+
}
77+
],
78+
"success": true,
79+
"errors": [],
80+
"messages": []
81+
}
82+
```

‎src/content/docs/log-explorer/custom-dashboards.mdx‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
pcx_content_type: reference
3+
title: Custom dashboards
4+
sidebar:
5+
order: 3
6+
---
7+
8+
Custom dashboards allow you to create tailored dashboards to monitor application security, performance, and usage. You can create monitors for ongoing monitoring of a previous incident, use them to identify indicators of suspicious activity, and access templates to help you get started.
9+
10+
:::note
11+
Enterprise customers can create up to 100 dashboards.
12+
13+
Customers on Pro and Business plans can create up to 5 dashboards.
14+
:::
15+
16+
Dashboards provide a visual interface that displays key metrics and analytics, helping you monitor and analyze data efficiently. Different dashboards serve different purposes. For example, a security dashboard tracks attack attempts and threats, a performance dashboard monitors API latency and uptime, and a usage dashboard analyzes traffic patterns and user behavior.
17+
18+
Different metrics serve distinct roles in providing insights into your application's performance. For example, total HTTP requests offer an overview of traffic volume, while average response time helps assess application speed. Additionally, usage metrics such as traffic patterns and user behavior provide insight into how users interact with your application. These metrics together enable you to spot trends, identify problems, and make informed, data-driven decisions.
19+
20+
## Create a new dashboard
21+
22+
To create a new dashboard:
23+
24+
1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com/login) and select your account.
25+
2. Go to **Log Explorer** > **Dashboards**.
26+
27+
When creating a dashboard, you have two options: building one from scratch or using a pre-designed template.
28+
29+
- A **templates** provide a faster way to set up a dashboard with commonly used metrics and charts. They are useful for standard use cases, such as monitoring security threats, API performance, or bot traffic. Templates help you get started quickly while still allowing modifications to fit your requirements.
30+
- On the other hand, **from-scratch dashboard** gives you full control over its structure, allowing you to choose the exact datasets, metrics, and visualizations that fit your needs. This approach is ideal if you have specific monitoring goals or need a highly customized view of your data.
31+
32+
Choosing between these options depends on whether you need a quick setup with predefined insights or a fully customized dashboard tailored to your unique analysis needs.
33+
34+
### Create a dashboard from scratch
35+
36+
When creating a dashboard from scratch, select the option **Create new**. You can follow the instructions in the following sections to start adding charts to your dashboard.
37+
38+
#### Create a new chart
39+
40+
To create a new chart, select **Add chart**. There are two ways to create a chart:
41+
42+
- **Use a prompt**: Enter a query like `Compare status code ranges over time.` The AI model decides the most appropriate visualization and constructs your chart configuration.
43+
- **Customize your chart**: Select the chart elements manually, including the chart type, title, dataset to query, metrics, and filters. This option gives you full control over your chart's structure.
44+
45+
Refer to the following sections for more information about the charts, datasets, fields, metrics, and filters available.
46+
47+
##### Chart types
48+
49+
The available chart types include:
50+
51+
- **Timeseries**: Displays trends over time, enabling comparisons across multiple series.
52+
- **Categorical**: Compares proportions across different series.
53+
- **Stat**: Highlights a single value, showing its delta and sparkline for quick insights.
54+
- **Percentage**: Represents one value as a percentage of another.
55+
- **Top N**: Identifies the highest-ranking values for a given attribute.
56+
57+
##### Datasets and metrics
58+
59+
The available metrics and filters vary based on the dataset you want to use. For example, when using the HTTP Requests dataset, you can select **origin response duration** as a metric. You can then choose your preferred aggregation method for that metric, such as total, median, or quantiles. The following table outlines the datasets, fields, and available metrics:
60+
61+
62+
| Dataset | Field | Definition | Metrics |
63+
|-----------------|-----------------|------------|---------|
64+
| HTTP Requests | Requests | The number of requests sent by a client to a server over the HTTP protocol. | Total |
65+
| | DNS Response Time | The time taken for a DNS query to be resolved, measured from when a request is made to when a response is received. | Total, Average, Median, 95th percentile, 99th percentile |
66+
| | Time to First Byte | The duration from when a request is made to when the first byte of the response is received from the server. | Total, Average, Median, 95th percentile, 99th percentile |
67+
| | Bytes returned to the Client | The amount of data (in bytes) sent from the server to the client in response to requests. | Total, Average, Median, 95th percentile, 99th percentile |
68+
| | Number of visits | Unique visits or sessions to a website or application. | Total |
69+
| | Origin response duration | The time taken by the origin server to process and respond to a request. | Total, Average, Median, 95th percentile, 99th percentile |
70+
| Security Events | Security events | Actions taken by Application Security products such as WAF and Bot Management. | Total |
71+
72+
##### Filters
73+
74+
You can also adjust the scope of your analytics by entering filter conditions. This allows you to focus on the most relevant data.
75+
76+
1. Select **Add filter**.
77+
2. Select a **field**, an **operator**, and a **value**. For example, to filter events by source IP address, select the _Source IP_ field, select the _equals_ operator, and enter the IP address.
78+
3. Select **Apply**.
79+
80+
### Create a dashboard from a template
81+
82+
Alternatively, you can choose to create your dashboard using a pre-designed dashboard template. The templates available are:
83+
84+
- **Bot monitoring**: Allows you to identify automated traffic accessing your website.
85+
- **API Security**: Allows you to monitor data transfers and exceptions for API endpoints in your application.
86+
- **Account takeover**: Allows you to monitor login attempts, usage of leaked credentials, and account takeover attacks.
87+
- **API Performance**: Allows you to view timing data for API endpoints in your application, along with error rates.
88+
- **Performance monitoring**: Allows you to identify slow hosts and paths on your origin server, and view time to first byte metrics over time.
89+
90+
Choose one of the templates and select **Use template**.
91+
92+
## Edit a dashboard or chart
93+
94+
After creating your dashboard, to view your saved dashboards, select **Back to all dashboards** to access the full list. Regardless of the way you choose to create your dashboard, you can always edit existing charts and add new ones as needed.
95+
96+
## Further analysis
97+
98+
For each chart, you can:
99+
100+
- Review related traffic in [Security Analytics](/waf/analytics/security-analytics/).
101+
- Explore detailed logs in [Log Explorer](/logs/log-explorer/).
102+
103+
This ensures deeper insights into your application's security, performance, and usage patterns.

‎src/content/docs/log-explorer/index.mdx‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
title: Log Explorer
3+
pcx_content_type: overview
4+
sidebar:
5+
order: 1
6+
---
7+
8+
import { Description, Feature, RelatedProduct } from "~/components"
9+
10+
<Description>
11+
Store and explore your Cloudflare logs directly within the Cloudflare dashboard or API.
12+
</Description>
13+
14+
Log Explorer is Cloudflare's native observability and forensics product that enables security teams and developers to analyze, investigate, and monitor issues directly from the Cloudflare dashboard, without the expense and complexity of forwarding logs to third-party tools.
15+
16+
Log Explorer provides access to Cloudflare logs with all the context available within the Cloudflare platform. You can monitor security and performance issues with custom dashboards or investigate and troubleshoot issues with log search. Benefits include:
17+
18+
- **Reduced cost and complexity**: Drastically reduce the expense and operational overhead associated with forwarding, storing, and analyzing terabytes of log data in external tools.
19+
- **Faster detection and triage**: Access Cloudflare-native logs directly, eliminating cumbersome data pipelines and the ingest lags that delay critical security insights.
20+
- **Accelerated investigations with full context**: Investigate incidents with Cloudflare's unparalleled contextual data, accelerating your analysis and understanding of "What exactly happened?" and "How did it happen?"
21+
- **Minimal recovery time**: Seamlessly transition from investigation to action with direct mitigation capabilities via the Cloudflare platform.
22+
23+
## Features
24+
25+
<Feature header="Log Search" href="/log-explorer/log-search/">
26+
Explore your Cloudflare logs directly within the Cloudflare dashboard or [API](/log-explorer/api/).
27+
</Feature>
28+
29+
<Feature header="Custom dashboards" href="/log-explorer/custom-dashboards/">
30+
Design customized views for tracking application security, performance, and usage metrics.
31+
</Feature>
32+
33+
<Feature header="Manage datasets" href="/log-explorer/manage-datasets/">
34+
Manage the data you want to store within Log Explorer.
35+
</Feature>
36+
37+
<Feature header="API" href= "/log-explorer/api/">
38+
Manage configuration and perform queries via the API.
39+
</Feature>
40+
41+
## Related products
42+
43+
<RelatedProduct header="Logpush" href="/logs/" product="logs">
44+
Forward Cloudflare logs to third-party tools for debugging, identifying configuration adjustments, and creating analytics dashboards.
45+
</RelatedProduct>
46+
47+
<RelatedProduct header="Analytics" href="/analytics/" product="analytics">
48+
Visualize the metadata collected by our products in the Cloudflare dashboard.
49+
</RelatedProduct>

0 commit comments

Comments
 (0)