Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions public/__redirects
Original file line number Diff line number Diff line change
Expand Up @@ -961,10 +961,10 @@
/logs/logpush/azure/ /logs/logpush/logpush-job/enable-destinations/azure/ 301
/logs/logpush/google-cloud-storage/ /logs/logpush/logpush-job/enable-destinations/google-cloud-storage/ 301
/logs/logpush/logpush-configuration-api/ /logs/logpush/logpush-job/enable-destinations/ 301
/logs/logpush/logpush-configuration-api/understanding-logpush-api/ /logs/get-started/api-configuration/ 301
/logs/logpush/logpush-configuration-api/understanding-logpush-api/ /logs/logpush/logpush-job/api-configuration/ 301
/logs/logpush/logpush-dashboard/ /logs/logpush/logpush-job/enable-destinations/ 301
/logs/logpush/s3-compatible-endpoints/ /logs/logpush/logpush-job/enable-destinations/s3-compatible-endpoints/ 301
/logs/reference/logpush-api-configuration/ /logs/get-started/api-configuration/ 301
/logs/reference/logpush-api-configuration/ /logs/logpush/logpush-job/api-configuration/ 301
/logs/reference/logpush-api-configuration/filters/ /logs/logpush/logpush-job/filters/ 301
# Non-slashed version is being used in the Cloudflare dashboard
/logs/reference/logpush-api-configuration/examples/example-logpush-curl/ /logs/logpush/examples/example-logpush-curl/ 301
Expand All @@ -981,6 +981,7 @@
/logs/reference/filters/ /logs/logpush/logpush-job/filters/ 301
/logs/reference/log-output-options/ /logs/logpush/logpush-job/log-output-options/ 301
/logs/reference/custom-fields/ /logs/logpush/logpush-job/custom-fields/ 301
/logs/get-started/api-configuration/ /logs/logpush/logpush-job/api-configuration/ 301

# magic-firewall
/magic-firewall/reference/examples/ /magic-firewall/how-to/add-rules/ 301
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ You can manage your Cloudflare Logpush service from the command line using cURL.

Before getting started, review the following documentation:

* [API configuration](/logs/get-started/api-configuration/)
* [API configuration](/logs/logpush/logpush-job/api-configuration/)

:::note

Expand All @@ -34,7 +34,7 @@ The examples below are for zone-scoped datasets. Account-scoped datasets should

### Parameters

* **destination\_conf** - Refer to [Destination](/logs/get-started/api-configuration/#destination) for details.
* **destination\_conf** - Refer to [Destination](/logs/logpush/logpush-job/api-configuration/#destination) for details.

### Response

Expand Down Expand Up @@ -93,15 +93,15 @@ When using Sumo Logic, you may find it helpful to have [Live Tail](https://help.
### Parameters

* **name** (optional) - We suggest using your domain name as the job name; the name cannot be changed after the job is created.
* **destination\_conf** - Refer to [Destination](/logs/get-started/api-configuration/#destination) for details.
* **destination\_conf** - Refer to [Destination](/logs/logpush/logpush-job/api-configuration/#destination) for details.
* **dataset** - The category of logs you want to receive. Refer to [Log fields](/logs/reference/log-fields/) for the full list of supported datasets; this parameter cannot be changed after the job is created.
* **output\_options** (optional) - Refer to [Log Output Options](/logs/logpush/logpush-job/log-output-options/).
* Typically includes the desired fields and timestamp format.
* Set the timestamp format to `RFC 3339` (`&timestamps=rfc3339`) for:
* Google BigQuery usage.
* Automated timestamp parsing within Sumo Logic; refer to [timestamps from Sumo Logic](https://help.sumologic.com/03Send-Data/Sources/04Reference-Information-for-Sources/Timestamps%2C-Time-Zones%2C-Time-Ranges%2C-and-Date-Formats) for details.
* **ownership\_challenge** - Challenge token required to prove destination ownership.
* **kind** (optional) - Used to differentiate between Logpush and Edge Log Delivery jobs. Refer to [Kind](/logs/get-started/api-configuration/#kind) for details.
* **kind** (optional) - Used to differentiate between Logpush and Edge Log Delivery jobs. Refer to [Kind](/logs/logpush/logpush-job/api-configuration/#kind) for details.
* **filter** (optional) - Refer to [Filters](/logs/logpush/logpush-job/filters/) for details.

### Response
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/logs/logpush/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ sidebar:

import { FeatureTable, LinkButton } from "~/components"

Logpush delivers logs in batches as quickly as possible, with no minimum batch size, potentially delivering files more than once per minute. This capability enables Cloudflare to provide information almost in real time, in smaller file sizes. Users can configure the batch size [using the API](/logs/get-started/api-configuration/#max-upload-parameters) for improved control in case the log destination has specific requirements.
Logpush delivers logs in batches as quickly as possible, with no minimum batch size, potentially delivering files more than once per minute. This capability enables Cloudflare to provide information almost in real time, in smaller file sizes. Users can configure the batch size [using the API](/logs/logpush/logpush-job/api-configuration/#max-upload-parameters) for improved control in case the log destination has specific requirements.

Logpush does not offer storage or search functionality for logs; its primary aim is to send logs as quickly as they arrive.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
pcx_content_type: concept
title: API configuration
sidebar:
order: 42
order: 2

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ import { LinkButton } from "~/components"

Edge Log Delivery allows customers to send logs directly from Cloudflare’s edge to their destination of choice. You can configure the maximum interval for your log batches between 30 seconds and five minutes. However, you cannot specify a minimum interval for log batches, meaning that log files may be sent in shorter intervals than the maximum specified. Compared to Logpush, Edge Log Delivery sends logs with lower latency, more frequently, and in smaller batches.

Edge Log Delivery is only available for HTTP request logs. Refer to the [API configuration](/logs/get-started/api-configuration/#kind) page for steps on how to configure a job to use Edge Log Delivery.
Edge Log Delivery is only available for HTTP request logs. Refer to the [API configuration](/logs/logpush/logpush-job/api-configuration/#kind) page for steps on how to configure a job to use Edge Log Delivery.

Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ When you are done entering the destination details, select **Continue**.

10. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

11. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ When you are done entering the destination details, select **Continue**.

9. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ When you are done entering the destination details, select **Continue**.

9. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ When you are done entering the destination details, select **Continue**.

10. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

11. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,14 @@ Cloudflare expects that the endpoint is available over HTTPS, using a trusted ce

9. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.

## Manage via API

To create a Logpush job, make a `POST` request to the [Logpush job creation endpoint URL](/logs/get-started/api-configuration/) with the appropriate parameters.
To create a Logpush job, make a `POST` request to the [Logpush job creation endpoint URL](/logs/logpush/logpush-job/api-configuration/) with the appropriate parameters.

The supported parameters are as follows:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ When you are done entering the destination details, select **Continue**.

9. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ To get started:
- Zone ID
- Destination access details for your cloud service provider

2. Configure your Logpush job. For more information on how to configure a Logpush job, refer to [API configuration](/logs/get-started/api-configuration/).
2. Configure your Logpush job. For more information on how to configure a Logpush job, refer to [API configuration](/logs/logpush/logpush-job/api-configuration/).
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ When you are done entering the destination details, select **Continue**.

9. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.
Expand All @@ -99,7 +99,7 @@ r2://<BUCKET_PATH>/{DATE}?account-id=<ACCOUNT_ID>&access-key-id=<R2_ACCESS_KEY_I
```

* **dataset** - The category of logs you want to receive. Refer to [Log fields](/logs/reference/log-fields/) for the full list of supported datasets.
* **output\_options** (optional) - To configure fields, sample rate, and timestamp format, refer to [API configuration options](/logs/get-started/api-configuration/#options).
* **output\_options** (optional) - To configure fields, sample rate, and timestamp format, refer to [API configuration options](/logs/logpush/logpush-job/api-configuration/#options).

Example request using cURL:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ When you are done entering the destination details, select **Continue**.

9. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ When you are done entering the destination details, select **Continue**.
9. In **Advanced Options**, you can:

- Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
- Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
- Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
- Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Cloudflare Logpush supports pushing logs directly to Sumo Logic via the Cloudfla

9. In **Advanced Options**, you can:
* Choose the format of timestamp fields in your logs (`RFC3339`(default),`Unix`, or `UnixNano`).
* Select a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Select a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) for your logs or push a randomly-sampled percentage of logs.
* Enable redaction for `CVE-2021-44228`. This option will replace every occurrence of `${` with `x{`.

10. Select **Submit** once you are done configuring your logpush job.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Jobs in Logpush now have a new key, **output\_options**, which replaces **logpul

## Replace logpull\_options

Previously, Logpush jobs could be customized by specifying the list of fields, sampling rate, and timestamp format in **logpull\_options** as [URL-encoded parameters](/logs/get-started/api-configuration/#options). For example:
Previously, Logpush jobs could be customized by specifying the list of fields, sampling rate, and timestamp format in **logpull\_options** as [URL-encoded parameters](/logs/logpush/logpush-job/api-configuration/#options). For example:

```json
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ pcx_content_type: concept

You can use Logpush with Magic Firewall IDS to log detected risks:

1. Consult the [Logpush Destination docs](/logs/get-started/api-configuration#destination) to learn about what destinations Logpush supports. The documentation will also instruct you on how to correctly format the destination URL for Logpush.
1. Consult the [Logpush Destination docs](/logs/logpush/logpush-job/api-configuration/#destination) to learn about what destinations Logpush supports. The documentation will also instruct you on how to correctly format the destination URL for Logpush.

2. Follow the [Manage Lopush with cURL](/logs/logpush/examples/example-logpush-curl/) tutorial to validate your Logpush destination and define a Logpush job.

Expand All @@ -18,4 +18,4 @@ You can use Logpush with Magic Firewall IDS to log detected risks:

* When creating the Logpush job, the dataset field should equal `magic_ids_detections`.

* Timestamps by default are unixnano. Consult the [Logpush Options docs](/logs/get-started/api-configuration#options) to learn what format you can choose that will be compatible with your destination and/or expectations. Note that all options must be added *after* all fields you want from the Logpush job, akin to URL parameters.
* Timestamps by default are unixnano. Consult the [Logpush Options docs](/logs/logpush/logpush-job/api-configuration/#options) to learn what format you can choose that will be compatible with your destination and/or expectations. Note that all options must be added *after* all fields you want from the Logpush job, akin to URL parameters.
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ For more details, refer to [`pcx_content_type`](/style-guide/frontmatter/custom-

[Cache: Common Cf-Polished statuses](/images/polish/cf-polished-statuses/)

[Logpush: Logpush API configuration](/logs/get-started/api-configuration/)
[Logpush: Logpush API configuration](/logs/logpush/logpush-job/api-configuration/)

## Template

Expand Down
4 changes: 2 additions & 2 deletions src/content/docs/workers/observability/logs/logpush.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ To create a Logpush job in the Cloudflare dashboard:

### Via cURL

The following example sends Workers logs to R2. For more configuration options, refer to [Enable destinations](/logs/logpush/logpush-job/enable-destinations/) and [API configuration](/logs/get-started/api-configuration/) in the Logs documentation.
The following example sends Workers logs to R2. For more configuration options, refer to [Enable destinations](/logs/logpush/logpush-job/enable-destinations/) and [API configuration](/logs/logpush/logpush-job/api-configuration/) in the Logs documentation.

```bash
curl "https://api.cloudflare.com/client/v4/accounts/<ACCOUNT_ID>/logpush/jobs" \
Expand All @@ -69,7 +69,7 @@ curl "https://api.cloudflare.com/client/v4/accounts/<ACCOUNT_ID>/logpush/jobs" \
}' | jq .
```

In Logpush, you can configure [filters](/logs/logpush/logpush-job/filters/) and a [sampling rate](/logs/get-started/api-configuration/#sampling-rate) to have more control of the volume of data that is sent to your configured destination. For example, if you only want to receive logs for requests that did not result in an exception, add the following `filter` JSON property below `output_options`:
In Logpush, you can configure [filters](/logs/logpush/logpush-job/filters/) and a [sampling rate](/logs/logpush/logpush-job/api-configuration/#sampling-rate) to have more control of the volume of data that is sent to your configured destination. For example, if you only want to receive logs for requests that did not result in an exception, add the following `filter` JSON property below `output_options`:

`"filter":"{\"where\": {\"key\":\"Outcome\",\"operator\":\"!eq\",\"value\":\"exception\"}}"`

Expand Down
Loading