-
Notifications
You must be signed in to change notification settings - Fork 10k
[Pipelines, Queues, Radar] Use DashButton component #24932
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,7 +8,7 @@ head: | |
| content: Configure HTTP endpoint | ||
| --- | ||
|
|
||
| import { Render, PackageManagers } from "~/components"; | ||
| import { Render, PackageManagers, DashButton } from "~/components"; | ||
|
|
||
| Pipelines support data ingestion over HTTP. When you create a new pipeline using the default settings you will receive a globally scalable ingestion endpoint. To ingest data, make HTTP POST requests to the endpoint. | ||
|
|
||
|
|
@@ -54,12 +54,16 @@ $ npx wrangler pipelines create [PIPELINE-NAME] --r2-bucket [R2-BUCKET-NAME] --r | |
| Once authentication is turned on, you will need to include a Cloudflare API token in your request headers. | ||
|
|
||
| ### Get API token | ||
| 1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com) and select your account. | ||
| 2. Navigate to your [API Keys](https://dash.cloudflare.com/profile/api-tokens). | ||
| 3. Select **Create Token**. | ||
| 4. Choose the template for Workers Pipelines. Select **Continue to summary** > **Create token**. Make sure to copy the API token and save it securely. | ||
|
|
||
| 1. In the Cloudflare dashboard, go to the **Account API tokens** page. | ||
|
|
||
| <DashButton url="/?to=/:account/api-tokens" /> | ||
|
|
||
| 2. Select **Create Token**. | ||
| 3. Choose the template for Workers Pipelines. Select **Continue to summary** > **Create token**. Make sure to copy the API token and save it securely. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Note: I couldn't find any template for Workers Pipelines... |
||
|
|
||
| ### Making authenticated requests | ||
|
|
||
| Include the API token you created in the previous step in the headers for your request: | ||
|
|
||
| ```sh | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,7 +12,7 @@ tags: | |
| - SQL | ||
| --- | ||
|
|
||
| import { Render, PackageManagers, Details, WranglerConfig } from "~/components"; | ||
| import { Render, PackageManagers, Details, WranglerConfig, DashButton } from "~/components"; | ||
|
|
||
| In this tutorial, you will learn how to build a data lake of website interaction events (clickstream data), using Pipelines. | ||
|
|
||
|
|
@@ -406,17 +406,21 @@ Now, you can access the application at the deployed URL. When you click on the ` | |
|
|
||
| ## 9. View the data in R2 | ||
|
|
||
| You can view the data in the R2 bucket. If you are not signed in to the Cloudflare dashboard, sign in and navigate to the [R2 overview](https://dash.cloudflare.com/?to=/:account/r2/overview) page. | ||
| To view the data in the R2 bucket: | ||
|
|
||
| Open the bucket you configured for your pipeline in Step 3. You can see files, representing the clickstream data. These files are newline delimited JSON files. Each row in a file represents one click event. Download one of the files, and open it in your preferred text editor to see the output: | ||
| 1. In the Cloudflare dashboard, go to R2's **Overview** page. | ||
|
|
||
| ```json | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Only change in code block was indentation. |
||
| {"timestamp":"2025-04-06T16:24:29.213Z","session_id":"1234567890abcdef","user_id":"user965","event_data":{"event_id":673,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:29.213Z","product_id":2},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:30.436Z","session_id":"1234567890abcdef","user_id":"user998","event_data":{"event_id":787,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:30.436Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:31.330Z","session_id":"1234567890abcdef","user_id":"user22","event_data":{"event_id":529,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:31.330Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:31.879Z","session_id":"1234567890abcdef","user_id":"user750","event_data":{"event_id":756,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:31.879Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:33.978Z","session_id":"1234567890abcdef","user_id":"user333","event_data":{"event_id":467,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:33.978Z","product_id":6},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| ``` | ||
| <DashButton url="/?to=/:account/r2/overview" /> | ||
|
|
||
| 2. Open the bucket you configured for your pipeline in Step 3. You can see files, representing the clickstream data. These files are newline delimited JSON files. Each row in a file represents one click event. Download one of the files, and open it in your preferred text editor to see the output: | ||
|
|
||
| ```json | ||
| {"timestamp":"2025-04-06T16:24:29.213Z","session_id":"1234567890abcdef","user_id":"user965","event_data":{"event_id":673,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:29.213Z","product_id":2},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:30.436Z","session_id":"1234567890abcdef","user_id":"user998","event_data":{"event_id":787,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:30.436Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:31.330Z","session_id":"1234567890abcdef","user_id":"user22","event_data":{"event_id":529,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:31.330Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:31.879Z","session_id":"1234567890abcdef","user_id":"user750","event_data":{"event_id":756,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:31.879Z","product_id":4},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| {"timestamp":"2025-04-06T16:24:33.978Z","session_id":"1234567890abcdef","user_id":"user333","event_data":{"event_id":467,"event_type":"product_view","page_url":"https://<URL>.workers.dev/","timestamp":"2025-04-06T16:24:33.978Z","product_id":6},"device_info":{"browser":"Chrome","os":"Linux","device":"Mobile","userAgent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Mobile Safari/537.36"},"referrer":""} | ||
| ``` | ||
|
|
||
| ## 10. Optional: Connect a query engine to your R2 bucket and query the data | ||
| Once you have collected the raw events in R2, you might want to query the events, to answer questions such as "how many events occurred?". You can connect a query engine, such as MotherDuck, to your R2 bucket. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,17 +11,22 @@ description: Use the dashboard to send messages to a queue. | |
|
|
||
| --- | ||
|
|
||
| import { DashButton } from "~/components"; | ||
|
|
||
| Sending messages from the dashboard allows you to debug Queues or queue consumers without a producer Worker. | ||
|
|
||
| To send messages from the dashboard: | ||
|
|
||
| 1. Log in to the [Cloudflare dashboard](https://dash.cloudflare.com) and select your account. | ||
| 2. Select **Workers & Pages** > **Queues**. | ||
| 3. Select the queue to send a message to. | ||
| 4. Select the **Messages** tab. | ||
| 5. Select **Send message**. | ||
| 6. Enter your message. You can choose your message content type by selecting the **Text** or **JSON** tabs. Alternatively, select the **Upload a file** button or drag a file over the textbox to upload a file as a message. | ||
| 7. Select **Send message**. | ||
| 1. In the Cloudflare dashboard, go to the **Queues** page. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instructions were outdated. |
||
|
|
||
| <DashButton url="/?to=/:account/workers/queues" /> | ||
|
|
||
| 2. Select the queue to send a message to. | ||
| 3. Select the **Messages** tab. | ||
| 4. Select **Send**. | ||
| 5. Choose your message **Content Type**: _Text_ or _JSON_. | ||
| 5. Enter your message. Alternatively, drag a file over the textbox to upload a file as a message. | ||
| 6. Select **Send**. | ||
|
|
||
| Your message will be sent to the queue. | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switched from user API tokens to account API tokens. Shouldn't have any impact.