Skip to content

Commit 1bd2613

Browse files
committed
Added pipelines changelog
1 parent 9bd62e4 commit 1bd2613

File tree

1 file changed

+39
-0
lines changed

1 file changed

+39
-0
lines changed
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
title: Pipelines now supports SQL transformations and Apache Iceberg
3+
description: Transform streaming data with SQL and write to Apache Iceberg tables in R2
4+
date: 2025-09-25T13:00:00
5+
products:
6+
- pipelines
7+
hidden: true
8+
---
9+
10+
import { LinkCard } from "~/components";
11+
12+
Today, we're launching the new [Cloudflare Pipelines](/pipelines/): a streaming data platform that ingests events, transforms them with [SQL](/pipelines/sql-reference/select-statements/), and writes to [R2](/r2/) as [Apache Iceberg](https://iceberg.apache.org/) tables or Parquet files.
13+
14+
Pipelines can receive events via [HTTP endpoints](/pipelines/streams/writing-to-streams/#send-via-http) or [Worker bindings](/pipelines/streams/writing-to-streams/#send-via-workers), transform them with SQL, and deliver to R2 with exactly-once guarantees. This makes it easy to build analytics-ready warehouses for server logs, mobile application events, IoT telemetry, or clickstream data without managing streaming infrastructure.
15+
16+
For example, here's a pipeline that ingests clickstream events and filters out bot traffic while extracting domain information:
17+
18+
```sql
19+
INSERT into events_table
20+
SELECT
21+
user_id,
22+
lower(event) AS event_type,
23+
to_timestamp_micros(ts_us) AS event_time,
24+
regexp_match(url, '^https?://([^/]+)')[1] AS domain,
25+
url,
26+
referrer,
27+
user_agent
28+
FROM events_json
29+
WHERE event = 'page_view'
30+
AND NOT regexp_like(user_agent, '(?i)bot|spider');
31+
```
32+
33+
Get started by creating a pipeline in the dashboard or running a single command in [Wrangler](/workers/wrangler/):
34+
35+
```bash
36+
npx wrangler pipelines setup
37+
```
38+
39+
Check out our [getting started guide](/pipelines/getting-started/) to learn how to create a pipeline that delivers events to an [Iceberg table](/r2/data-catalog/) you can query with R2 SQL. Read more about today's announcement in our [blog post](https://blog.cloudflare.com/cloudflare-data-platform).

0 commit comments

Comments
 (0)