Small changes

Author: maheshwarip

src/assets/images/pipelines/shards.png

@@ -5,32 +5,46 @@ sidebar:
   order: 1
 ---
 
-Cloudflare Pipelines let you ingest data from a source, and deliver to a destination. It's built for high volume, real time data streams. Each pipeline can ingest up to 100 MB/s of data, via HTTP or a Worker, and load the data as files in an R2 bucket.
+Cloudflare Pipelines let you ingest data from a source, and deliver to a sink. It's built for high volume, real time data streams. Each pipeline can ingest up to 100 MB/s of data, via HTTP or a Worker, and load the data as files in an R2 bucket.
 
 This guide explains how a pipeline works.
 
 ![Pipelines Architecture](~/assets/images/pipelines/architecture.png)
 
 ## Supported sources, data formats, and sinks
-Pipelines supports ingestion via [HTTP](/pipelines/build-with-pipelines/http), or from a [Cloudflare Worker](/workers/), using the [Pipelines Workers API](/pipelines/build-with-pipelines/workers-apis).
 
-A pipeline can ingest JSON-serializable records.
+### Sources
+Pipelines supports the following sources:
+* [HTTP Clients](/pipelines/build-with-pipelines/http), with optional authentication and CORS settings
+* [Cloudflare Worker](/workers/), using the [Pipelines Workers API](/pipelines/build-with-pipelines/workers-apis)
 
-Finally, Pipelines supports writing data to [R2 Object Storage](/r2/). Ingested data is written to output files. These files are optionally compressed, and then delivered to an R2 bucket. Output files are generated as newline delimited JSON files (`ndjson`). The filename of each output file is prefixed by the event date and time, to make querying the data more efficient. For example, an output fle might be named like this: `event_date=2025-04-03/hr=15/01JQY361X75TMYSQZGWC6ZDMR2.json.gz`. Each line in an output file maps to a single record ingested by a pipeline.
+Multiple sources can be active on a single pipeline simultaneously. For example, you can create a pipeline which accepts data from both a Worker, and via HTTP. Multiple workers can be configured to send data to the same pipeline. There is no limit to the number of source clients.
 
-We plan to support more sources, data formats, and sinks, in the future.
+### Data format
+Pipelines can ingest JSON serializable records.
 
-## Data durability, and the lifecycle of a request
-If you make a request to send data to a pipeline, and receive a successful response, we guarantee that the data will be delivered to your configured destination.
+### Sinks
+Pipelines supports delivering data into [R2 Object Storage](/r2/). Ingested data is delivered as newline delimited JSON files (`ndjson`), with optional compression. Multiple pipelines can be configured to deliver data to the same R2 bucket.
 
-Any data sent to a pipeline is durably committed to storage. Pipelines use [SQLite backed Durable Objects](/durable-objects/best-practices/access-durable-objects-storage/#sqlite-storage-backend) as a buffer for ingested records. A pipeline will only return a response after data has been successfully stored.
+## Data durability
+Pipelines are designed to be reliable. Data sent to a pipeline should be delivered successfully to the configured R2 bucket, provided that the [R2 API credentials associated with a pipeline](/r2/api/s3/tokens/) remain valid.
 
-Ingested data continues to be stored, until a sufficiently large batch of data has accumulated. Batching is useful to reduce the number of output files written out to R2. [Batch sizes are customizable](/pipelines/build-with-pipelines/output-settings/#customize-batch-behavior), in terms of data volume, rows, or time.
+Each pipeline maintains a storage buffer. Requests to send data to a pipeline receive a successful response only after the data is committed to this storage buffer.
 
-When a batch has reached its target size, the batch entire is written out to a file. The file is optionally compressed, and is delivered to an R2 bucket. Any transient failures, such as network failures, are automatically retried.
+Ingested data accumulates, until a sufficiently [large batch of data](/pipelines/build-with-pipelines/output-settings/#customize-batch-behavior) has been filled. Once the batch reaches its target size, the entire batch of data is converted to a file and delivered to R2.
+
+Transient failures, such as network connectivity issues, are automatically retried.
+
+However, if the [R2 API credentials associated with a pipeline](/r2/api/s3/tokens/) expire or are revoked, data delivery will fail. In this scenario, some data might continue to accumulate in the buffers, but the pipeline will eventually start rejecting requests.
 
 ## How a Pipeline handles updates
-Data delivery is guaranteed even while updating an existing pipeline. Updating an existing pipeline effectively creates a new deployment, including all your previously configured options. Requests are gracefully re-routed to the new pipeline. The old pipeline continues to write data into your destination. Once the old pipeline is fully drained, it is spun down.
+Pipelines update without dropping records. Updating an existing pipeline effectively creates a new instance of the pipeline. Requests are gracefully re-routed to the new instance. The old instance continues to write data into your configured sink. Once the old instance is fully drained, it is spun down.
+
+This means that updates might take a few minutes to go into effect. For example, if you update a pipeline's sink, previously ingested data might continue to be delivered into the old sink.
 
 ## What if I send too much data? Do Pipelines communicate backpressure?
-If you send too much data, the pipeline will communicate backpressure by returning a 429 response to HTTP requests, or throwing an error if using the Workers API. Refer to the [limits](/pipelines/platform/limits) to learn how much volume a single pipeline can support.
+If you send too much data, the pipeline will communicate backpressure by returning a 429 response to HTTP requests, or throwing an error if using the Workers API. Refer to the [limits](/pipelines/platform/limits) to learn how much volume a single pipeline can support. You might see 429 responses if you are sending too many requests, or sending too much data.
+
+If you are consistently seeing backpressure from your pipeline, consider the following strategies:
+* Increase the [shard count](/pipelines/build-with-pipelines/shards), to increase the maxiumum throughput of your pipeline.
+* Send data to a second pipeline if you receive an error. You can setup multiple pipelines to write to the same R2 bucket.

src/content/docs/pipelines/concepts/how-pipelines-work.mdx

@@ -23,4 +23,8 @@ Many of these limits will increase during Pipelines' public beta period.
 | Maximum batch duration	                      | 300s 	                                          							|
 
 ## What happens if I exceed the requests per second or throughput limits?
-Consistently exceeding the requests per second or throughpht limits may result in requests being rejected by your Pipeline, with a status code of 429. This status code indicates that your Pipeline is unable to keep up with the volume.
+If you consistently exceed the requests per second or throughput limits, your pipeline might not be able to keep up with the load. The pipeline will communicate backpressure by returning a 429 response to HTTP requests, or throwing an error if using the Workers API.
+
+If you are consistently seeing backpressure from your pipeline, consider the following strategies:
+* Increase the [shard count](/pipelines/build-with-pipelines/shards), to increase the maxiumum throughput of your pipeline.
+* Send data to a second pipeline if you receive an error. You can setup multiple pipelines to write to the same R2 bucket.