You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Use a Node version manager like [Volta](https://volta.sh/) or [nvm](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions. Wrangler requires a Node version of 16.17.0 or later.
34
+
Use a Node version manager like [Volta](https://volta.sh/) or [nvm](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions.
35
+
36
+
Wrangler requires a Node version of 16.17.0 or later.
35
37
:::
36
38
37
39
## 1. Set up authentication
38
40
39
-
You'll need API tokens to interact with Cloudflare services.
41
+
You will need API tokens to interact with Cloudflare services.
40
42
41
43
<Steps>
42
44
1. In the Cloudflare dashboard, go to the **R2 object storage** page.
45
+
43
46
<DashButtonurl="/?to=/:account/r2/overview" />
44
47
45
48
2. Select **Manage API tokens**.
@@ -63,6 +66,7 @@ export WRANGLER_R2_SQL_AUTH_TOKEN= #paste your token here
63
66
```
64
67
65
68
If this is your first time using Wrangler, make sure to login.
69
+
66
70
```bash
67
71
npx wrangler login
68
72
```
@@ -83,18 +87,19 @@ Create an R2 bucket:
83
87
84
88
<Steps>
85
89
1. In the Cloudflare dashboard, go to the **R2 object storage** page.
90
+
86
91
<DashButtonurl="/?to=/:account/r2/overview" />
87
92
88
93
2. Select **Create bucket**.
89
94
90
-
3. Enter the bucket name: r2-sql-demo
95
+
3. Enter the bucket name: `r2-sql-demo`
91
96
92
97
4. Select **Create bucket**.
93
98
</Steps>
94
99
</TabItem>
95
100
</Tabs>
96
101
97
-
## 2. Enable R2 Data Catalog
102
+
## 3. Enable R2 Data Catalog
98
103
99
104
<TabssyncKey='CLIvDash'>
100
105
<TabItemlabel='Wrangler CLI'>
@@ -112,9 +117,10 @@ When you run this command, take note of the "Warehouse". You will need these lat
112
117
113
118
<Steps>
114
119
1. In the Cloudflare dashboard, go to the **R2 object storage** page.
120
+
115
121
<DashButtonurl="/?to=/:account/r2/overview" />
116
122
117
-
2. Select the bucket: r2-sql-demo.
123
+
2. Select the bucket: `r2-sql-demo`.
118
124
119
125
3. Switch to the **Settings** tab, scroll down to **R2 Data Catalog**, and select **Enable**.
120
126
@@ -125,20 +131,22 @@ When you run this command, take note of the "Warehouse". You will need these lat
125
131
126
132
127
133
:::note
128
-
Copy the warehouse (ACCOUNTID_BUCKETNAME) and paste it in the `export` below. We'll use it later in the tutorial.
134
+
Copy the warehouse (ACCOUNTID_BUCKETNAME) and paste it in the `export` below. We will use it later in the tutorial.
129
135
:::
130
136
131
137
```bash
132
138
export$WAREHOUSE= #Paste your warehouse here
133
139
```
134
140
135
-
## 3. Create the data Pipeline
141
+
## 4. Create the data Pipeline
136
142
137
143
<TabssyncKey='CLIvDash'>
138
144
<TabItemlabel='Wrangler CLI'>
139
-
### 1. Create the Pipeline Stream
145
+
146
+
### 4.1. Create the Pipeline Stream
140
147
141
148
First, create a schema file called `demo_schema.json` with the following `json` schema:
149
+
142
150
```json
143
151
{
144
152
"fields": [
@@ -148,7 +156,7 @@ First, create a schema file called `demo_schema.json` with the following `json`
148
156
]
149
157
}
150
158
```
151
-
Next, crete the stream we'll use to ingest events to:
159
+
Next, create the stream we will use to ingest events to:
This creates a `sink` configuration that will write to the Iceberg table demo.first_table in your R2 Data Catalog every 30 seconds. Pipelines automatically appends an `__ingest_ts` column that is used to partition the table by `DAY`
219
+
This creates a `sink` configuration that will write to the Iceberg table `demo.first_table` in your R2 Data Catalog every 30 seconds. Pipelines automatically appends an `__ingest_ts` column that is used to partition the table by `DAY`.
211
220
:::
212
221
213
-
### 3. Create the Pipeline
222
+
### 4.3. Create the Pipeline
214
223
215
-
Pipelines are SQL statements read data from the stream, does some work, and writes it to the sink
224
+
Pipelines are SQL statements that reads data from the stream, does some work, and writes it to the sink.
216
225
217
226
```bash
218
227
npx wrangler pipelines create demo_pipeline \
219
228
--sql "INSERT INTO demo_sink SELECT * FROM demo_stream WHERE numbers > 5;"
220
229
```
221
230
:::note
222
-
Note that there is a filter on this statement that will only send events where `numbers` is greater than 5
231
+
Note that there is a filter on this statement that will only send events where `numbers` is greater than 5.
223
232
:::
224
233
225
234
</TabItem>
226
235
<TabItemlabel='Dashboard'>
227
236
<Steps>
228
-
1. In the Cloudflare dashboard, go to **Pipelines** > **Pipelines**.
237
+
1. In the Cloudflare dashboard, go to the Pipelines page.
238
+
229
239
<DashButtonurl="/?to=/:account/pipelines" />
230
240
231
241
2. Select **Create Pipeline**.
@@ -272,7 +282,7 @@ Note that there is a filter on this statement that will only send events where `
272
282
- Select **Create Pipeline**
273
283
274
284
8. :::note
275
-
Note the **HTTP Ingest Endpoint URL** from the output. This is the endpoint you'll use to send data to your pipeline.
285
+
Note the **HTTP Ingest Endpoint URL** from the output. This is the endpoint you will use to send data to your pipeline.
276
286
:::
277
287
278
288
</Steps>
@@ -287,7 +297,7 @@ export STREAM_ENDPOINT= #the http ingest endpoint from the output (see example b
287
297
288
298
## 5. Send some data
289
299
290
-
Next, let's send some events to our stream:
300
+
Next, send some events to our stream:
291
301
292
302
```curl
293
303
curl -X POST "$STREAM_ENDPOINT" \
@@ -314,16 +324,19 @@ curl -X POST "$STREAM_ENDPOINT" \
314
324
}
315
325
]'
316
326
```
327
+
317
328
This will send 4 events in one `POST`. Since our Pipeline is filtering out records with `numbers` less than 5, `user_id``3` and `4` should not appear in the table. Feel free to change values and send more events.
318
329
319
330
## 6. Query the table with R2 SQL
320
331
321
-
After you've sent your events to the stream, it will take about 30 seconds for the data to show in the table since that's what we configured our `roll interval` to be in the Sink.
332
+
After you have sent your events to the stream, it will take about 30 seconds for the data to show in the table, since that is what we configured our `roll interval` to be in the Sink.
Copy file name to clipboardExpand all lines: src/content/docs/r2-sql/index.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ description: A distributed SQL engine for R2 Data Catalog
16
16
R2 SQL is in public beta, and any developer with an R2 subscription can start using it. Currently, outside of standard R2 storage and operations, you will not be billed for your use of R2 SQL. We will update [the pricing page](/r2-sql/platform/pricing) and provide at least 30 days notice before enabling billing.
17
17
:::
18
18
19
-
R2 SQL is Cloudflare's serverless, distributed, analytics query engine for querying [Apache Iceberg](https://iceberg.apache.org/) tables stored in [R2 data catalog](https://developers.cloudflare.com/r2/data-catalog/). R2 SQL is designed to efficiently query large amounts of data by automatically utilizing file pruning, Cloudflare's distributed compute, and R2 object storage.
19
+
R2 SQL is Cloudflare's serverless, distributed, analytics query engine for querying [Apache Iceberg](https://iceberg.apache.org/) tables stored in [R2 data catalog](/r2/data-catalog/). R2 SQL is designed to efficiently query large amounts of data by automatically utilizing file pruning, Cloudflare's distributed compute, and R2 object storage.
Copy file name to clipboardExpand all lines: src/content/docs/r2-sql/query-data.mdx
+11-7Lines changed: 11 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ import {
11
11
} from"~/components";
12
12
13
13
:::note
14
-
R2 SQL is currently in open beta
14
+
R2 SQL is currently in open beta.
15
15
:::
16
16
17
17
Learn how to:
@@ -32,11 +32,11 @@ Create an [API token](https://dash.cloudflare.com/profile/api-tokens) with:
32
32
- Access to R2 storage (**minimum**: read-only)
33
33
- Access to R2 SQL (**minimum**: read-only)
34
34
35
-
Wrangler now supports the environment variable `WRANGLER_R2_SQL_AUTH_TOKEN` which you can `export` your token as.
35
+
Wrangler now supports the environment variable `WRANGLER_R2_SQL_AUTH_TOKEN` which you can use to `export` your token.
36
36
37
37
### Create API token via API
38
38
39
-
To create an API token programmatically for use with R2 SQL, you'll need to specify R2 SQL, R2 Data Catalog, and R2 storage permission groups in your [Access Policy](/r2/api/tokens/#access-policy).
39
+
To create an API token programmatically for use with R2 SQL, you will need to specify R2 SQL, R2 Data Catalog, and R2 storage permission groups in your [Access Policy](/r2/api/tokens/#access-policy).
Copy file name to clipboardExpand all lines: src/content/docs/r2-sql/reference/limitations-best-practices.mdx
+18-19Lines changed: 18 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,29 +8,28 @@ sidebar:
8
8
9
9
---
10
10
11
-
# R2 SQL Limitations and Best Practices
12
-
13
11
## Overview
14
12
15
-
R2 SQL is in public beta, limitations and best practices will change over time.
13
+
:::note
14
+
R2 SQL is in public beta. Limitations and best practices will change over time.
15
+
:::
16
16
17
17
R2 SQL is designed for querying **partitioned** Apache Iceberg tables in your R2 data catalog. This document outlines the supported features, limitations, and best practices of R2 SQL.
18
18
19
-
20
19
## Quick Reference
21
20
22
-
| Feature | Supported | Notes |
23
-
| :---- | :---- | :---- |
24
-
| Basic SELECT | Yes | Columns, \*|
25
-
| Aggregation functions | No | No COUNT, AVG, etc. |
26
-
| Single table FROM | Yes | Note, aliasing not supported|
0 commit comments