Skip to content

Commit f4b1a20

Browse files
committed
Added more details to the get started guide
1 parent a45e523 commit f4b1a20

File tree

1 file changed

+66
-1
lines changed

1 file changed

+66
-1
lines changed

src/content/docs/pipelines/get-started.mdx

Lines changed: 66 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,69 @@ head:
88
content: Get started
99
---
1010

11-
Test
11+
import { Render, PackageManagers } from "~/components";
12+
13+
:::note
14+
15+
Pipelines is in **public beta**, and any developer with a [paid Workers plan](/workers/platform/pricing/#workers) can start using Pipelines immediately.
16+
17+
Pipelines let you ingest real time streaming data, and store it in R2. By following this guide, you'll create your first Pipeline, connect it to your R2 bucket, and post data to it via HTTP.
18+
19+
:::
20+
21+
## Prerequisites
22+
23+
To use Pipelines, you will need:
24+
25+
<Render file="prereqs" product="workers" />
26+
27+
### 1. Enable Pipelines
28+
29+
TODO
30+
31+
### 2. Install Wrangler
32+
TODO
33+
34+
## 1. Setup an R2 bucket to use as a destination
35+
Pipelines is built to ingest data, and store it in an R2 bucket. Create a bucket, following our [Get Started Guide for R2](https://developers.cloudflare.com/r2/get-started/) if you need to. Save the bucket name; you'll need it for the next sep.
36+
37+
## 2. Create a Pipeline
38+
To create a pipeline using Wrangler, run this command in a shell. Specify the name of your pipeline, as well as the name of the R2 bucket you created in Step 1.
39+
40+
```sh
41+
npx wrangler pipelines create <PIPELINE-NAME> --r2 <R2-BUCKET-NAME>
42+
```
43+
44+
Choose a descrpitive name for your Pipeline, related to the type of events you intend to ingest. You cannot change the Pipeline name after you have set it.
45+
46+
TODO: Pipeline name limitations
47+
48+
Once your pipeline is created, you'll receive an HTTP endpoint which you can post data to. You should see output which resembles the below:
49+
50+
```sh
51+
🌀 Authorizing R2 bucket "<R2-BUCKET-NAME>"
52+
🌀 Creating pipeline named "<PIPELINE-NAME>"
53+
✅ Successfully created pipeline <PIPELINE-NAME> with ID <PIPELINE-ID>
54+
55+
You can now send data to your pipeline with:
56+
curl "https://<PIPELINE-ID>.pipelines.cloudflare.com/" -d '[{ ...JSON_DATA... }]'
57+
```
58+
59+
## 3. Post data to your pipeline
60+
61+
Use a curl command in your terminal to post an array of JSON objects to the endpoint you received in Step 1.
62+
63+
```sh
64+
curl -H "Content-Type:application/json" \
65+
-d '[{"account_id":"test", "other_data": "test"},{"account_id":"test","other_data": "test2"}]' \
66+
<HTTP-endpoint>
67+
```
68+
69+
Once the data has been successfully accepted by the Pipeline, you'll receive a success message.
70+
71+
Pipelines handles batching the data, so you can continue posting data to the Pipeline. Once a batch is filled up, the data will be partitioned by date, and written to your R2 bucket.
72+
73+
## 4. Verify in R2
74+
Navigate to the dashboard for the R2 bucket that you created in step 1. You should see a prefix for today's date. Click through, and you'll see a file created containing the JSON data you sent in Step 3.
75+
76+
By completing this guide, you've got a pipeline, with an HTTP endpoint as a source, and an R2 bucket as a destination.

0 commit comments

Comments
 (0)