Improved output settings

Author: maheshwarip

src/content/docs/pipelines/build-with-pipelines/output-settings.mdx

@@ -13,43 +13,31 @@ import { Render, PackageManagers } from "~/components";
 Pipelines convert a stream of records into output files, and deliver the files to an R2 bucket in your account. This guide details how you can change the output destination, and how to customize batch settings to generate query ready files.
 
 ## Configure an R2 bucket as a destination
-To create or update a Pipeline using Wrangler, run the following command in a terminal:
+To create or update a pipeline using Wrangler, run the following command in a terminal:
 
 ```sh
 npx wrangler pipelines create [PIPELINE-NAME] --r2-bucket [R2-BUCKET-NAME]
 ```
 
-After running this command, you'll be prompted to authorize Cloudflare Workers Pipelines to create R2 API tokens on your behalf. These tokens are required by your Pipeline. Your Pipeline uses the tokens when loading data into your bucket. You can approve the request through the browser link which will open automatically.
+After running this command, you'll be prompted to authorize Cloudflare Workers Pipelines to create an R2 API token on your behalf. Your pipeline uses the R2 API token to load data into your bucket. You can approve the request through the browser link which will open automatically.
 
-If you prefer not to authenticate this way, you may pass your [R2 API Tokens](/r2/api/s3/tokens/) to Wrangler:
+If you prefer not to authenticate this way, you may pass your [R2 API Token](/r2/api/s3/tokens/) to Wrangler:
 ```sh
 npx wrangler pipelines create [PIPELINE-NAME] --r2 [R2-BUCKET-NAME] --r2-access-key-id [ACCESS-KEY-ID] --r2-secret-access-key [SECRET-ACCESS-KEY]
 ```
 
-## Deliver data to a prefix
-You can specify an optional prefix for all the output files stored in your specified R2 bucket.
-
-To modify the prefix for an existing Pipeline using Wrangler:
-```sh
-npx wrangler pipelines update <pipeline-name> --r2-prefix "test"
-```
-
-All the output records generated by your pipeline will be stored under the prefix "test", and will look like this:
-```sh
-- test/event_date=2024-09-06/hr=15/37db9289-15ba-4e8b-9231-538dc7c72c1e-15.json.gz
-- test/event_date=2024-09-06/hr=15/37db9289-15ba-4e8b-9231-538dc7c72c1e-15.json.gz
-```
-
 ## File format and compression
 Output files are generated as Newline Delimited JSON files (`ndjson`). Each line in an output file maps to a single record.
 
 By default, output files are compressed in the `gzip` format. Compression can be turned off using the `--compression`  flag:
 ```sh
-npx wrangler pipelines update <pipeline-name> --compression none
+npx wrangler pipelines update [PIPELINE-NAME] --compression none
 ```
 
+Output files are named using a [UILD](/https://github.com/ulid/spec) slug, followed by an extension.
+
 ## Customize batch behavior
-When configuring your Pipeline, you can define how records are batched before they are delivered to R2. Batches of records are written out to a single output file.
+When configuring your pipeline, you can define how records are batched before they are delivered to R2. Batches of records are written out to a single output file.
 
 Batching can:
 1. Reduce the number of output files written to R2, and thus reduce the [cost of writing data to R2](/r2/pricing/#class-a-operations)
@@ -61,22 +49,24 @@ There are three ways to define how ingested data is batched:
 2. `batch-max-rows`: The maximum number of rows or events in a batch before data is written. Default, and maximum, is 10,000 rows.
 3. `batch-max-seconds`: The maximum duration of a batch before data is written, in seconds. Default is 15 seconds, maximum is 300 seconds.
 
-Pipelines batch definitions are hints. A pipeline will follow these hints closely, but batches will not be exact.
+Batch definitions are hints. A pipeline will follow these hints closely, but batches might not be exact.
 
 All three batch definitions work together. Whichever limit is reached first triggers the delivery of a batch.
 
-For example, a `batch-max-mb` = 100 MB and a `batch-max-seconds` = 100 means that if 100 MB of events are posted to the Pipeline, the batch will be delivered. However, if it takes longer than 100 seconds for 100 MB of events to be posted, a batch of all the messages that were posted during those 100 seconds will be created.
+For example, a `batch-max-mb` = 100 MB and a `batch-max-seconds` = 100 means that if 100 MB of events are posted to the pipeline, the batch will be delivered. However, if it takes longer than 100 seconds for 100 MB of events to be posted, a batch of all the messages that were posted during those 100 seconds will be created.
 
 ### Defining batch settings using Wrangler
-To update the batch settings for an existing Pipeline using Wrangler, run the following command in a terminal
+You can use the following batch settings flags while creating or updating a pipeline:
+* `--batch-max-mb`
+* `--batch-max-rows`
+* `--batch-max-seconds`
 
+For example:
 ```sh
 npx wrangler pipelines update [PIPELINE-NAME] --batch-max-mb 100 --batch-max-rows 10000 --batch-max-seconds 300
 ```
 
-### Batch settings
-
-You can configure the following batch-level settings to adjust how Pipelines create a batch:
+#### Batch size limits
 
 | Setting                                   | Default     | Minimum   | Maximum     |
 | ----------------------------------------- | ----------- | --------- | ----------- |
@@ -86,16 +76,28 @@ You can configure the following batch-level settings to adjust how Pipelines cre
 
 
 ## Deliver partitioned data
-Partitioning organizes data into directories based on specific fields to improve query performance. It helps by reducing the amount of data scanned for queries, enabling faster reads.
+Partitioning organizes data into directories based on specific fields to improve query performance. Partitions reduce the amount of data scanned for queries, enabling faster reads.
 
 :::note
-By default, all Pipelines partition data by event date and time. This will be customizable in the future.
+By default, Pipelines partition data by event date and time. This will be customizable in the future.
 :::
 
 Output files are prefixed with event date and hour. For example, the output from a Pipeline in your R2 bucket might look like this:
 ```sh
-- event_date=2024-09-06/hr=15/37db9289-15ba-4e8b-9231-538dc7c72c1e-15.json.gz
-- event_date=2024-09-06/hr=15/37db9289-15ba-4e8b-9231-538dc7c72c1e-15.json.gz
+- event_date=2025-04-01/hr=15/01JQWBZCZBAQZ7RJNZHN38JQ7V.json.gz
+- event_date=2025-04-01/hr=15/01JQWC16FXGP845EFHMG1C0XNW.json.gz
 ```
 
+## Deliver data to a prefix
+You can specify an optional prefix for all the output files stored in your specified R2 bucket, using the flag `--r2-prefix`.
+
+For example:
+```sh
+npx wrangler pipelines update [PIPELINE-NAME] --r2-prefix test
+```
 
+After running the above command, the output files generated by your pipeline will be stored under the prefix "test". Files will remain partitioned. Your output will look like this:
+```sh
+- test/event_date=2025-04-01/hr=15/01JQWBZCZBAQZ7RJNZHN38JQ7V.json.gz
+- test/event_date=2025-04-01/hr=15/01JQWC16FXGP845EFHMG1C0XNW.json.gz
+```