You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Use a Node version manager like [Volta](https://volta.sh/) or [nvm](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions.
@@ -47,13 +47,13 @@ You will need API tokens to interact with Cloudflare services.
47
47
48
48
2. Select **Manage API tokens**.
49
49
50
-
3. Select **Create API token**.
50
+
3. Select **Create User API token**.
51
51
52
52
4. Select the **R2 Token** text to edit your API token name.
53
53
54
54
5. Under **Permissions**, choose the **Admin Read & Write** permission.
55
55
56
-
6. Select **Create API Token**.
56
+
6. Select **Create User API Token**.
57
57
58
58
7. Note the **Token value**.
59
59
@@ -99,8 +99,6 @@ Create an R2 bucket:
99
99
</TabItem>
100
100
</Tabs>
101
101
102
-
## 3. Enable R2 Data Catalog
103
-
104
102
<TabssyncKey='CLIvDash'>
105
103
<TabItemlabel='Wrangler CLI'>
106
104
@@ -138,12 +136,12 @@ Copy the warehouse (ACCOUNTID_BUCKETNAME) and paste it in the `export` below. We
138
136
export$WAREHOUSE= #Paste your warehouse here
139
137
```
140
138
141
-
## 4. Create the data Pipeline
139
+
## 3. Create the data Pipeline
142
140
143
141
<TabssyncKey='CLIvDash'>
144
142
<TabItemlabel='Wrangler CLI'>
145
143
146
-
### 4.1. Create the Pipeline Stream
144
+
### 3.1. Create the Pipeline Stream
147
145
148
146
First, create a schema file called `demo_schema.json` with the following `json` schema:
149
147
@@ -201,7 +199,7 @@ Input Schema:
201
199
└────────────┴────────┴────────────┴──────────┘
202
200
```
203
201
204
-
### 4.2. Create the Pipeline Sink
202
+
### 3.2. Create the Pipeline Sink
205
203
206
204
Create a sink that writes data to your R2 bucket as Apache Iceberg tables:
This creates a `sink` configuration that will write to the Iceberg table `demo.first_table` in your R2 Data Catalog every 30 seconds. Pipelines automatically appends an `__ingest_ts` column that is used to partition the table by `DAY`.
220
218
:::
221
219
222
-
### 4.3. Create the Pipeline
220
+
### 3.3. Create the Pipeline
223
221
224
222
Pipelines are SQL statements that reads data from the stream, does some work, and writes it to the sink.
225
223
@@ -295,7 +293,7 @@ export STREAM_ENDPOINT= #the http ingest endpoint from the output (see example b
295
293
</Tabs>
296
294
297
295
298
-
## 5. Send some data
296
+
## 4. Send some data
299
297
300
298
Next, send some events to our stream:
301
299
@@ -327,7 +325,7 @@ curl -X POST "$STREAM_ENDPOINT" \
327
325
328
326
This will send 4 events in one `POST`. Since our Pipeline is filtering out records with `numbers` less than 5, `user_id``3` and `4` should not appear in the table. Feel free to change values and send more events.
329
327
330
-
## 6. Query the table with R2 SQL
328
+
## 5. Query the table with R2 SQL
331
329
332
330
After you have sent your events to the stream, it will take about 30 seconds for the data to show in the table, since that is what we configured our `roll interval` to be in the Sink.
Copy file name to clipboardExpand all lines: src/content/docs/r2-sql/query-data.mdx
+9-5Lines changed: 9 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,6 +8,9 @@ sidebar:
8
8
import {
9
9
Render,
10
10
LinkCard,
11
+
Tabs,
12
+
TabItem,
13
+
Steps
11
14
} from"~/components";
12
15
13
16
:::note
@@ -24,8 +27,8 @@ R2 SQL can currently be accessed via Wrangler commands or a REST API.
24
27
25
28
To query Apache Iceberg tables in R2 Data Catalog, you must provide a Cloudflare API token with R2 SQL, R2 Data Catalog, and R2 storage permissions.
26
29
27
-
### Create API token in the dashboard
28
-
30
+
<TabssyncKey='CLIvDash'>
31
+
<TabItemlabel='Dashboard'>
29
32
Create an [API token](https://dash.cloudflare.com/profile/api-tokens) with:
30
33
31
34
- Access to R2 Data Catalog (**minimum**: read-only)
@@ -34,8 +37,8 @@ Create an [API token](https://dash.cloudflare.com/profile/api-tokens) with:
34
37
35
38
Wrangler now supports the environment variable `WRANGLER_R2_SQL_AUTH_TOKEN` which you can use to `export` your token.
36
39
37
-
### Create API token via API
38
-
40
+
</TabItem>
41
+
<TabItemlabel='Via API'>
39
42
To create an API token programmatically for use with R2 SQL, you will need to specify R2 SQL, R2 Data Catalog, and R2 storage permission groups in your [Access Policy](/r2/api/tokens/#access-policy).
40
43
41
44
#### Example Access Policy
@@ -66,7 +69,8 @@ To create an API token programmatically for use with R2 SQL, you will need to sp
This creates a `sink` configuration that will write to the Iceberg table `fraud_detection.transactions` in your R2 Data Catalog every 30 seconds. Pipelines automatically appends an `__ingest_ts` column that is used to partition the table by `DAY`.
0 commit comments