You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this tutorial, you will learn how to ingest clickstream data from your website, using Pipelines. We'll also use Pipelines to load this data into an [R2 bucket](/r2/). You will also learn how to connect the bucket to MotherDuck. You will then query the data using MotherDuck.
18
+
In this tutorial, you will learn how to ingest clickstream data to a [R2 bucket](/r2)using Pipelines. You will use the Pipeline binding to send the clickstream data to the R2 bucket from your Worker. You will also learn how to connect the bucket to MotherDuck. You will then query the data using MotherDuck.
19
19
20
20
For this tutorial, you will build a landing page of an e-commerce website. A user can click on the view button to view the product details or click on the add to cart button to add the product to their cart.
21
21
22
-
We will use a pipeline to ingest these click events, and load into an R2 bucket. We'll then use MotherDuck to query the events.
23
-
24
22
## Prerequisites
25
23
26
24
1. A [MotherDuck](https://motherduck.com/) account.
@@ -50,7 +48,7 @@ Create a new Worker project by running the following commands:
50
48
product="workers"
51
49
params={{
52
50
category: "hello-world",
53
-
type: "Worker only",
51
+
type: "Worker + Assets",
54
52
lang: "TypeScript",
55
53
}}
56
54
/>
@@ -61,20 +59,9 @@ Navigate to the `e-commerce-pipelines` directory:
61
59
cd e-commerce-pipelines
62
60
```
63
61
64
-
## 2. Create the frontend
65
-
66
-
Using Static Assets, you can serve the frontend of your application from your Worker. To use Static Assets, you need to add the required bindings to your `wrangler.toml` file.
62
+
## 2. Update the frontend
67
63
68
-
<WranglerConfig>
69
-
70
-
```toml
71
-
[assets]
72
-
directory = "public"
73
-
```
74
-
75
-
</WranglerConfig>
76
-
77
-
Next, create a `public` directory in the root of your project. Add an `index.html` file to the `public` directory. The `index.html` file should contain the following HTML code:
64
+
Using Static Assets, you can serve the frontend of your application from your Worker. The above step creates a new Worker project with a default `public/index.html` file. Update the `public/index.html` file with the following HTML code:
78
65
79
66
<details>
80
67
<summary>Select to view the HTML code</summary>
@@ -308,41 +295,72 @@ The `handleClick` function does the following:
308
295
- Logs any errors that occur.
309
296
310
297
## 4. Create an R2 bucket
311
-
We'll create a new R2 bucket to use as the sink for our pipeline. Create a new r2 bucket `clickstream-data` using the [Wrangler CLI](/workers/wrangler/):
298
+
299
+
You will create a new R2 bucket to use as the sink for our pipeline. Create a new r2 bucket `clickstream-data` using the [Wrangler CLI](/workers/wrangler/):
312
300
313
301
```sh
314
302
npx wrangler r2 bucket create clickstream-data
315
303
```
316
304
317
305
## 5. Create a pipeline
318
-
You need to create a new pipeline and connect it to the R2 bucket we created in the previous step.
306
+
You need to create a new pipeline and connect it to the R2 bucket you created in the previous step.
319
307
320
308
Create a new pipeline `clickstream-pipeline` using the [Wrangler CLI](/workers/wrangler/):
When you run the command, you will be prompted to authorize Cloudflare Workers Pipelines to create R2 API tokens on your behalf. These tokens are required by your Pipeline. Your Pipeline uses these tokens when loading data into your bucket. You can approve the request through the browser link which will open automatically.
327
315
316
+
:::note
317
+
The above command creates a pipeline using two optional flags: `--compression none`, and `--batch-max-seconds 5`.
318
+
319
+
With these flags, your pipeline will deliver an uncompressed file of data to your R2 bucket every 5 seconds.
320
+
321
+
These flags are useful for testing, but we recommend keeping the default settings in a production environment.
322
+
:::
323
+
328
324
```output
329
-
🌀 Authorizing R2 bucket clickstream-data
330
-
🌀 Creating pipeline named "clickstream-pipeline"
331
-
✅ Successfully created pipeline "clickstream-pipeline" with id <PIPELINES_ID>
325
+
✅ Successfully created Pipeline "clickstream-pipeline" with ID <PIPELINE_ID>
We've setup the frontend of our application to make a call to the `/api/clickstream` route, everytime the user clicks on one of the
336
354
337
-
The front-end of the application makes a call to the `/api/clickstream` endpoint to send the clickstream data to your pipeline. The `/api/clickstream` endpoint is handled by a Worker in the `src/index.ts` file.
355
+
You have setup the frontend of your application to make a call to the`/api/clickstream` route, everytime the user clicks on one of the buttons. The application makes a call to the `/api/clickstream` endpoint to send the clickstream data to your pipeline. The `/api/clickstream` endpoint is handled by a Worker in the `src/index.ts` file.
338
356
339
357
You will use the pipelines binding to send the clickstream data to your pipeline. In your `wrangler` file, add the following bindings:
340
358
341
359
<WranglerConfig>
342
360
343
361
```toml
344
362
[[pipelines]]
345
-
binding = "MY_PIPELINE"
363
+
binding = "PIPELINE"
346
364
pipeline = "clickstream-pipeline"
347
365
```
348
366
@@ -352,7 +370,7 @@ Next, update the type in the `worker-configuration.d.ts` file. Add the following
352
370
353
371
```ts title="worker-configuration.d.ts"
354
372
interfaceEnv {
355
-
MY_PIPELINE:Pipeline;
373
+
PIPELINE:Pipeline;
356
374
}
357
375
```
358
376
@@ -366,7 +384,7 @@ export default {
366
384
if (pathname==="/api/clickstream"&&method==="POST") {
367
385
const body = (awaitrequest.json()) as { data:any };
368
386
try {
369
-
awaitenv.MY_PIPELINE.send([body.data]);
387
+
awaitenv.PIPELINE.send([body.data]);
370
388
returnnewResponse("OK", { status: 200 });
371
389
} catch (error) {
372
390
console.error(error);
@@ -472,4 +490,4 @@ This project serves as a foundation for building scalable, data-driven applicati
472
490
473
491
For your next steps, consider exploring more advanced querying techniques with MotherDuck, implementing real-time analytics, or integrating additional Cloudflare services to further optimize your application's performance and security.
474
492
475
-
You can find the source code of the application in the [GitHub repository](https://github.com/harshil1712/e-commerce-clickstream).
493
+
You can find the source code of the application in the [GitHub repository](https://github.com/harshil1712/cf-pipelines-bindings-demo).
Copy file name to clipboardExpand all lines: src/content/docs/pipelines/tutorials/send-data-from-client/index.mdx
+5-18Lines changed: 5 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
---
2
-
updated: 2025-04-06
2
+
updated: 2025-04-09
3
3
difficulty: Intermediate
4
4
content_type: 📝 Tutorial
5
5
pcx_content_type: tutorial
@@ -49,7 +49,7 @@ Create a new Worker project by running the following commands:
49
49
product="workers"
50
50
params={{
51
51
category: "hello-world",
52
-
type: "Worker only",
52
+
type: "Worker + Assets",
53
53
lang: "TypeScript",
54
54
}}
55
55
/>
@@ -60,20 +60,9 @@ Navigate to the `e-commerce-pipelines-client-side` directory:
60
60
cd e-commerce-pipelines-client-side
61
61
```
62
62
63
-
## 2. Create the website frontend
63
+
## 2. Update the website frontend
64
64
65
-
Using [Workers Static Assets](/workers/static-assets/), you can serve the frontend of your application from your Worker. To use Static Assets, you need to add the required bindings to your `wrangler.toml` file.
66
-
67
-
<WranglerConfig>
68
-
69
-
```toml
70
-
[assets]
71
-
directory = "public"
72
-
```
73
-
74
-
</WranglerConfig>
75
-
76
-
Next, create a `public` directory and add an `index.html` file. The `index.html` file should contain the following HTML code:
65
+
Using Static Assets, you can serve the frontend of your application from your Worker. The above step creates a new Worker project with a default `public/index.html` file. Update the `public/index.html` file with the following HTML code:
77
66
78
67
<details>
79
68
<summary>Select to view the HTML code</summary>
@@ -237,7 +226,7 @@ Sources:
237
226
Format: JSON
238
227
Destination:
239
228
Type: R2
240
-
Bucket: apr-6
229
+
Bucket: clickstream-bucket
241
230
Format: newline-delimited JSON
242
231
Compression: NONE
243
232
Batch hints:
@@ -439,8 +428,6 @@ You can connect the bucket to MotherDuck in several ways, which you can learn ab
439
428
440
429
Before connecting the bucket to MotherDuck, you need to obtain the Access Key ID and Secret Access Key for the R2 bucket. You can find the instructions to obtain the keys in the [R2 API documentation](/r2/api/tokens/).
441
430
442
-
Before connecting the bucket to MotherDuck, you need to obtain the Access Key ID and Secret Access Key for the R2 bucket. You can find the instructions to obtain the keys in the [R2 API documentation](/r2/api/tokens/).
443
-
444
431
1. Log in to the MotherDuck dashboard and select your profile.
445
432
2. Navigate to the **Secrets** page.
446
433
3. Select the **Add Secret** button and enter the following information:
0 commit comments