Skip to content

Commit 41bbabf

Browse files
committed
Updated logo and text
1 parent f237dea commit 41bbabf

File tree

4 files changed

+20
-6
lines changed

4 files changed

+20
-6
lines changed
-1.98 KB
Loading

src/content/docs/pipelines/concepts/how-pipelines-work.mdx

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ sidebar:
55
order: 1
66
---
77

8-
Cloudflare Pipelines let you ingest data from a source, and deliver to a destination. It's built for high volume, real time data streams. Each Pipeline can ingest up to 100 MB/s of data, via HTTP or a Worker, and load the data as files in an R2 bucket.
8+
Cloudflare Pipelines let you ingest data from a source, and deliver to a destination. It's built for high volume, real time data streams. Each pipeline can ingest up to 100 MB/s of data, via HTTP or a Worker, and load the data as files in an R2 bucket.
99

10-
This guide explains how a Pipeline works.
10+
This guide explains how a pipeline works.
1111

1212
![Pipelines Architecture](~/assets/images/pipelines/architecture.png)
1313

@@ -16,7 +16,7 @@ Pipelines supports ingestion via [HTTP](/pipelines/build-with-pipelines/http), o
1616

1717
A pipeline can ingest JSON-serializable records.
1818

19-
Finally, Pipelines supports R2 as a sink. Ingested data is written to output files, compressed, and delivered to an R2 bucket. Output files are generated as newline delimited JSON files (`ndjson`). The filename of each output file is prefixed by the event date and time, to make querying the data more efficient. For example, an output fle might be named like this: `event_date=2025-04-03/hr=15/01JQY361X75TMYSQZGWC6ZDMR2.json.gz`. Each line in an output file maps to a single record ingested by a pipeline.
19+
Finally, Pipelines supports writing data to [R2 Object Storage](/r2/). Ingested data is written to output files, compressed, and delivered to an R2 bucket. Output files are generated as newline delimited JSON files (`ndjson`). The filename of each output file is prefixed by the event date and time, to make querying the data more efficient. For example, an output fle might be named like this: `event_date=2025-04-03/hr=15/01JQY361X75TMYSQZGWC6ZDMR2.json.gz`. Each line in an output file maps to a single record ingested by a pipeline.
2020

2121
We plan to support more sources, data formats, and sinks, in the future.
2222

@@ -27,7 +27,7 @@ Any data sent to a pipeline is durably committed to storage. Pipelines use [SQLi
2727

2828
Ingested data is buffered until a sufficiently large batch of data has accumulated. Batching is useful to reduce the number of output files written out to R2. [Batch sizes are customizable](/pipelines/build-with-pipelines/output-settings/#customize-batch-behavior), in terms of data volume, rows, or time.
2929

30-
Finally, the the batch of data is converted into output files, which are compressed, and delivered to the configured R2 bucket. Any transient failures, such as network failures, are automatically retried.
30+
Finally, when a batch has reached its target size, it is written out to a file. The file is compressed, and is delivered to the configured R2 bucket. Any transient failures, such as network failures, are automatically retried.
3131

3232
## How a Pipeline handles updates
3333
Data delivery is guaranteed even while updating an existing pipeline. Updating an existing pipeline effectively creates a new deployment, including all your previously configured options. Requests are gracefully re-routed to the new pipeline. The old pipeline continues to write data into your destination. Once the old pipeline is fully drained, it is spun down.

src/content/products/pipelines.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,5 @@ product:
88

99
meta:
1010
title: Cloudflare Pipelines Docs
11-
description: Ingest, transform, and store, real time data streams in R2.
11+
description: Ingest real time data streams, and load to R2.
1212
author: '@cloudflare'

src/icons/pipelines.svg

Lines changed: 15 additions & 1 deletion
Loading

0 commit comments

Comments
 (0)