diff --git a/src/content/docs/logs/get-started/api-configuration.mdx b/src/content/docs/logs/get-started/api-configuration.mdx index 8636180ee424d07..b8bba0257378c25 100644 --- a/src/content/docs/logs/get-started/api-configuration.mdx +++ b/src/content/docs/logs/get-started/api-configuration.mdx @@ -6,8 +6,6 @@ sidebar: --- -import { APIRequest } from "~/components"; - ## Endpoints The table below summarizes the job operations available for both Logpush and Edge Log Delivery jobs. Make sure that Account-scoped datasets use `/accounts/{account_id}` and Zone-scoped datasets use `/zone/{zone_id}`. For more information, refer to the [Log fields](/logs/reference/log-fields/) page. @@ -39,10 +37,11 @@ For concrete examples, refer to the tutorials in [Logpush examples](/logs/tutori The Logpush API requires credentials like any other Cloudflare API. - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` ## Ownership @@ -50,13 +49,15 @@ Before creating a new job, ownership of the destination must be proven. To issue an ownership challenge token to your destination: -?region=us-west-2", - }} -/> +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/ownership \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "destination_conf": "s3://?region=us-west-2" +}' +``` A challenge file will be written to the destination, and the filename will be in the response (the filename may be expressed as a path, if appropriate for your destination): @@ -142,13 +143,15 @@ For more information on the value for your cloud storage provider, consult the f To check if a destination is already in use: - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/validate/destination/exists \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "destination_conf": "s3://foo" +}' +``` Response @@ -179,30 +182,22 @@ The kind parameter cannot be used to update existing Logpush jobs. You can only ::: -", - destination_conf: "s3://?region=us-west-2", - dataset: "http_requests", - output_options: { - field_names: [ - "ClientIP", - "ClientRequesrHost", - "ClientRequestMethod", - " ClientRequestURI", - "EdgeEndTimestamp", - "EdgeResponseBytes", - "EdgeResponseStatus", - "EdgeStartTimestamp", - "RayID" - ], - timestamp_format: "rfc3339" - }, - kind: "edge" - }} -/> +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "name": "", + "destination_conf": "s3://?region=us-west-2", + "dataset": "http_requests", + "output_options": { + "field_names": ["ClientIP", "ClientRequesrHost", "ClientRequestMethod", " ClientRequestURI", "EdgeEndTimestamp", "EdgeResponseBytes", "EdgeResponseStatus", "EdgeStartTimestamp","RayID"], + "timestamp_format": "rfc3339" + }, + "kind": "edge" +}' +``` ## Options @@ -221,14 +216,16 @@ The **CVE-2021-44228** parameter can only be set through the API at this time. U To check if the selected **logpull\_options** are valid: - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/validate/origin \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "logpull_options": "fields=RayID,ClientIP,EdgeStartTimestamp×tamps=rfc3339&CVE-2021-44228=true", + "dataset": "http_requests" +}' +``` Response diff --git a/src/content/docs/logs/instant-logs.mdx b/src/content/docs/logs/instant-logs.mdx index 7f335633a951db9..ec824d6b9833dd8 100644 --- a/src/content/docs/logs/instant-logs.mdx +++ b/src/content/docs/logs/instant-logs.mdx @@ -5,7 +5,7 @@ sidebar: order: 116 --- -import { FeatureTable, APIRequest } from "~/components"; +import { FeatureTable } from "~/components"; Instant Logs allows Cloudflare customers to access a live stream of the traffic for their domain from the Cloudflare dashboard or from a command-line interface (CLI). Seeing data in real time allows you to investigate an attack, troubleshoot, debug or test out changes made to your network. Instant Logs is lightweight, simple to use and does not require any additional setup. @@ -66,16 +66,18 @@ Below we have three examples of filters: Example request using cURL: - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/edge/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "fields": "ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID", + "sample": 100, + "filter": "", + "kind": "instant-logs" +}' +``` Response: diff --git a/src/content/docs/logs/reference/custom-fields.mdx b/src/content/docs/logs/reference/custom-fields.mdx index 22b6d55103a12eb..c2bed6917f5e1aa 100644 --- a/src/content/docs/logs/reference/custom-fields.mdx +++ b/src/content/docs/logs/reference/custom-fields.mdx @@ -5,8 +5,6 @@ sidebar: order: 43 --- -import { APIRequest } from "~/components"; - The HTTP requests dataset includes most standard log information by default. However, if you need to capture additional request or response headers or cookies, you can use custom fields to tailor the logs to your specific needs Custom fields are configured per zone and, once set up, are enabled for all Logpush jobs in that zone that use the HTTP requests dataset and include the request headers, response headers, or cookie fields. You can log these fields in their raw form or as transformed values. @@ -74,27 +72,28 @@ Perform the following steps to create the rule: 1. Use the [List existing rulesets](/ruleset-engine/rulesets-api/view/#list-existing-rulesets) operation to check if there is already a ruleset for the `http_log_custom_fields` phase at the zone level (you can only have one zone ruleset per phase): - + ```bash + curl https://api.cloudflare.com/client/v4/zones/{zone_id}/rulesets \ + --header "X-Auth-Email: " \ + --header "X-Auth-Key: " + ``` If there is a ruleset for the `http_log_custom_fields` phase at the zone level, take note of the ruleset ID. 2. (Optional) If the response did not include a ruleset with `"kind": "zone"` and `"phase": "http_log_custom_fields"`, create the phase entry point ruleset using the [Create ruleset](/ruleset-engine/rulesets-api/create/) operation: - + ```bash + curl https://api.cloudflare.com/client/v4/zones/{zone_id}/rulesets \ + --header "X-Auth-Email: " \ + --header "X-Auth-Key: " \ + --header "Content-Type: application/json" \ + --data '{ + "name": "Zone-level phase entry point", + "kind": "zone", + "description": "This ruleset configures custom log fields.", + "phase": "http_log_custom_fields" + }' + ``` Take note of the ruleset ID included in the response. @@ -102,41 +101,43 @@ Perform the following steps to create the rule: The following example configures custom fields with the names of the HTTP request headers, HTTP response headers, and cookies you wish to include in Logpush logs: -" \ + --header "X-Auth-Key: " \ + --header "Content-Type: application/json" \ + --data '{ + "rules": [ + { + "action": "log_custom_field", + "expression": "true", + "description": "Set Logpush custom fields for HTTP requests", + "action_parameters": { + "request_fields": [ + { "name": "content-type" }, + { "name": "x-forwarded-for" } + ], + "transformed_request_fields": [ + {"name": "host"} ], - cookie_fields: [ - { name: "__ga" }, - { name: "accountNumber" }, - { name: "__cfruid" } - ] - } - } - ] - }} - roles={false} -/> + "response_fields": [ + { "name": "server" }, + { "name": "content-type" } + ], + "raw_response_fields": [ + { "name": "allow" } + ], + "cookie_fields": [ + { "name": "__ga" }, + { "name": "accountNumber" }, + { "name": "__cfruid" } + ] + } + } + ] + }' + ``` Example response: @@ -192,25 +193,27 @@ Perform the following steps to create the rule: Some headers sent from the origin — such as `set-cookie` — may have multiple values that you want to capture. You can use the Rulesets API to specify which headers should have all their values logged. - + } + ] +}' +``` Note that `preserve_duplicates` applies to both `response_fields` and `raw_response_fields`. If there are no transform rules that affect a header, including `preserve_duplicates` in either `response_fields` or `raw_response_fields` should achieve the same result. @@ -233,26 +236,22 @@ Next, include `Cookies`, `RequestHeaders`, `ResponseHeaders`, and/or `ResponseFi For example, consider the following request that creates a job that includes custom fields: -", - destination_conf: "s3://?region=us-west-2", - dataset: "http_requests", - output_options: { - field_names: [ - "RayID", - "EdgeStartTimestamp", - "Cookies", - "RequestHeaders", - "ResponseHeaders" - ], - timestamp_format: "rfc3339" - }, - ownership_challenge: "" - }} -/> +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "name": "", + "destination_conf": "s3://?region=us-west-2", + "dataset": "http_requests", + "output_options": { + "field_names": ["RayID", "EdgeStartTimestamp", "Cookies", "RequestHeaders", "ResponseHeaders"], + "timestamp_format": "rfc3339" + }, + "ownership_challenge": "" +}' +``` :::note[Note for Cloudflare Access users] diff --git a/src/content/docs/logs/reference/filters.mdx b/src/content/docs/logs/reference/filters.mdx index 18cbbd22431ebe3..942a2fedc3ea9c4 100644 --- a/src/content/docs/logs/reference/filters.mdx +++ b/src/content/docs/logs/reference/filters.mdx @@ -7,7 +7,7 @@ sidebar: --- -import { Render, APIRequest } from "~/components" +import { Render } from "~/components" The following table represents the comparison operators that are supported and example values. Filters are added as escaped JSON strings formatted as `{"key":"","operator":"","value":""}`. @@ -62,29 +62,24 @@ Filters can be set via API or the Cloudflare dashboard. Note that using a filter Here is an example request using cURL via API: -?region=us-west-2/" - }} -/> +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "name": "static-assets", + "output_options": { + "field_names": ["ClientIP", "EdgeStartTimestamp", "RayID"], + "sample_rate": 0.1, + "timestamp_format": "rfc3339" + "CVE-2021-44228": "true" + }, + "dataset": "http_requests", + "filter": "{\"where\":{\"and\":[{\"key\":\"ClientRequestPath\",\"operator\":\"contains\",\"value\":\"/static\"},{\"key\":\"ClientRequestHost\",\"operator\":\"eq\",\"value\":\"example.com\"}]}}", + "destination_conf": "s3://?region=us-west-2/" +}' +``` ### Dashboard diff --git a/src/content/docs/logs/tutorials/examples/example-logpush-curl.mdx b/src/content/docs/logs/tutorials/examples/example-logpush-curl.mdx index 2bc1d1976cb5095..910bff8f45035ad 100644 --- a/src/content/docs/logs/tutorials/examples/example-logpush-curl.mdx +++ b/src/content/docs/logs/tutorials/examples/example-logpush-curl.mdx @@ -6,8 +6,6 @@ sidebar: --- -import { APIRequest } from "~/components"; - You can manage your Cloudflare Logpush service from the command line using cURL. Before getting started, review the following documentation: @@ -24,13 +22,15 @@ The examples below are for zone-scoped datasets. Account-scoped datasets should ## Step 1 - Get ownership challenge -?region=us-west-2" - }} -/> +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/ownership \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "destination_conf": "s3://?region=us-west-2" +}' +``` ### Parameters @@ -65,30 +65,22 @@ When using Sumo Logic, you may find it helpful to have [Live Tail](https://help. ## Step 2 - Create a job -", - destination_conf: "s3://?region=us-west-2", - dataset: "http_requests", - output_options: { - field_names: [ - "ClientIP", - "ClientRequestHost", - "ClientRequestMethod", - "ClientRequestURI", - "EdgeEndTimestamp", - "EdgeResponseBytes", - "EdgeResponseStatus", - "EdgeStartTimestamp", - "RayID" - ], - timestamp_format: "rfc3339" - }, - ownership_challenge: "" - }} -/> +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "name": "", + "destination_conf": "s3://?region=us-west-2", + "dataset": "http_requests", + "output_options": { + "field_names": ["ClientIP", "ClientRequestHost", "ClientRequestMethod", "ClientRequestURI", "EdgeEndTimestamp","EdgeResponseBytes", "EdgeResponseStatus", "EdgeStartTimestamp", "RayID"], + "timestamp_format": "rfc3339" + }, + "ownership_challenge": "" +}' +``` ### Parameters @@ -134,10 +126,11 @@ In the response, you get a newly-created job ID. For example: Start by retrieving information about a specific job, using a job ID: - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs/{job_id} \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` ### Response @@ -167,20 +160,24 @@ Note that by default a job is not enabled (`"enabled": false`). If you do not remember your job ID, you can retrieve it using your zone ID: - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` Next, to enable the job, send an update request: - +```bash +curl --request PUT \ +https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs/{job_id} \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "enabled": true +}' +``` ### Response @@ -214,10 +211,11 @@ Once a job has been enabled and has started executing, the **last\_complete** fi ### Request to get job by ID and see **last\_complete** info - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs/{job_id} \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` ### Response @@ -245,10 +243,12 @@ Once a job has been enabled and has started executing, the **last\_complete** fi ## Optional - Delete a job - +```bash +curl --request DELETE \ +https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs/{job_id} \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` Be careful when deleting a job because this action cannot be reversed. @@ -267,10 +267,11 @@ Be careful when deleting a job because this action cannot be reversed. Retrieve a specific job, using the job ID: - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs/{job_id} \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` ### Response @@ -300,10 +301,11 @@ Retrieve a specific job, using the job ID: Retrieve all jobs for all datasets: - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` ### Response @@ -347,10 +349,11 @@ Retrieve all jobs for all datasets: If you want to add (or remove) fields, change the timestamp format, or enable protection against the `Log4j - CVE-2021-44228` vulnerability, first retrieve the current **output\_options** for your zone. - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs/{job_id} \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " +``` ### Response @@ -379,26 +382,19 @@ If you want to add (or remove) fields, change the timestamp format, or enable pr Next, edit the **output\_options** as desired and create a `PUT` request. The following example enables the **CVE-2021-44228** redaction option. - +```bash +curl --request PUT \ +https://api.cloudflare.com/client/v4/zones/{zone_id}/logpush/jobs/{job_id} \ +--header "X-Auth-Email: " \ +--header "X-Auth-Key: " \ +--header "Content-Type: application/json" \ +--data '{ + "output_options": { + "field_names": ["ClientIP", "ClientRequestHost", "ClientRequestMethod", "ClientRequestURI", "EdgeEndTimestamp","EdgeResponseBytes", "EdgeResponseStatus", "EdgeStartTimestamp", "RayID"], + "timestamp_format": "rfc3339" + } +} +``` Note that at this time, the **CVE-2021-44228** option is not available through the UI, and updating your Logpush job through the UI will remove this option. diff --git a/src/content/partials/logs/check-log-retention.mdx b/src/content/partials/logs/check-log-retention.mdx index 6a5b9c2e8254154..52a57570d1a1868 100644 --- a/src/content/partials/logs/check-log-retention.mdx +++ b/src/content/partials/logs/check-log-retention.mdx @@ -3,14 +3,14 @@ --- -import { TabItem, Tabs, APIRequest } from "~/components"; +import { TabItem, Tabs } from "~/components"; - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logs/control/retention/flag \ +--header "Authorization: Bearer " +``` diff --git a/src/content/partials/logs/disable-log-retention.mdx b/src/content/partials/logs/disable-log-retention.mdx index 3a89db7ac7c39e0..5e7b875bf2d01cf 100644 --- a/src/content/partials/logs/disable-log-retention.mdx +++ b/src/content/partials/logs/disable-log-retention.mdx @@ -3,17 +3,18 @@ --- -import { TabItem, Tabs, APIRequest } from "~/components"; +import { TabItem, Tabs } from "~/components"; - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logs/control/retention/flag \ +--header "Authorization: Bearer " \ +--header "Content-Type: application/json" \ +--data '{ + "flag": false +}' +``` diff --git a/src/content/partials/logs/enable-log-retention.mdx b/src/content/partials/logs/enable-log-retention.mdx index 7d9a9ac4a7a3e7c..1c0e242dce1a68b 100644 --- a/src/content/partials/logs/enable-log-retention.mdx +++ b/src/content/partials/logs/enable-log-retention.mdx @@ -3,17 +3,17 @@ --- -import { TabItem, Tabs, APIRequest } from "~/components"; +import { TabItem, Tabs } from "~/components"; - +```bash +curl https://api.cloudflare.com/client/v4/zones/{zone_id}/logs/control/retention/flag \ +--request POST \ +--header "Authorization: Bearer " \ +--header "Content-Type: application/json" \ +--data '{"flag": true}' +```