Skip to content

Commit 07f0c6a

Browse files
committed
Applying PCX Style Guide to Batching chapter.
1 parent 3ef2aab commit 07f0c6a

File tree

1 file changed

+13
-14
lines changed

1 file changed

+13
-14
lines changed

src/content/docs/pipelines/configuration/batching.mdx

Lines changed: 13 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -3,27 +3,26 @@ pcx_content_type: concept
33
title: Batching
44
sidebar:
55
order: 10
6-
76
---
87

9-
# Batching
10-
Pipelines automatically batches reqests that are received via HTTP or from a Worker. Batching is helpful to reduce the number of output files written to your destination, which can make them more efficient to query.
8+
Pipelines automatically batches requests that are received via HTTP or from a Worker. Batching helps reduce the number of output files written to your destination, which can make them more efficient to query.
119

1210
There are three ways to define how requests are batched:
13-
1. `batch-max-mb`: The maximum amount of data that will be batched, in megabytes. Default 10 MB, maximum 100 MB.
14-
2. `batch-max-rows`: The maximum number of rows or events in a batch before data is written. Default, and maximum, 10,000.
15-
3. `batch-max-seconds`: The maximum duration of a batch before data is written, in seconds. Default 15s. Maximum of 600s.
1611

17-
All three batch definitions work together. Whichever limit is reached first will trigger the delivery of a batch.
12+
1. `batch-max-mb`: The maximum amount of data that will be batched, in megabytes. Default is 10 MB, maximum is 100 MB.
13+
2. `batch-max-rows`: The maximum number of rows or events in a batch before data is written. Default, and maximum, is 10,000 rows.
14+
3. `batch-max-seconds`: The maximum duration of a batch before data is written, in seconds. Default is 15 seconds, maximum is 600 seconds.
15+
16+
All three batch definitions work together. Whichever limit is reached first triggers the delivery of a batch.
1817

19-
For example, a `batch-max-mb` = 100 MB and a `batch-max-seconds` = 600 means that if 100 MB of events are posted to the Pipeline, the batch will be delivered. However, if it takes longer than 600s for 100 MB of events to be posted, a batch of all the messages that were posted during those 600s will be created and delivered.
18+
For example, a `batch-max-mb` = 100 MB and a `batch-max-seconds` = 600 means that if 100 MB of events are posted to the Pipeline, the batch will be delivered. However, if it takes longer than 600 seconds for 100 MB of events to be posted, a batch of all the messages that were posted during those 600 seconds will be created and delivered.
2019

2120
## Batch settings
22-
The following batch-level settings can be configured to adjust how Pipelines creates a batch:
2321

24-
| Setting | Default | Minimum | Maximum |
25-
| ----------------------------------------- | ----------- | --------- | ------------ |
26-
| Maximum Batch Size `batch-max-mb` | 10 MB | 0.001 MB | 100 MB |
27-
| Maximum Batch Timeout `batch-max-seconds` | 15 seconds | 0 seconds | 600 seconds |
28-
| Maximum Batch Rows `batch-max-rows` | 10,000 rows | 1 row | 10,000 rows |
22+
You can configure the following batch-level settings to adjust how Pipelines create a batch:
2923

24+
| Setting | Default | Minimum | Maximum |
25+
| ----------------------------------------- | ----------- | --------- | ----------- |
26+
| Maximum Batch Size `batch-max-mb` | 10 MB | 0.001 MB | 100 MB |
27+
| Maximum Batch Timeout `batch-max-seconds` | 15 seconds | 0 seconds | 600 seconds |
28+
| Maximum Batch Rows `batch-max-rows` | 10,000 rows | 1 row | 10,000 rows |

0 commit comments

Comments
 (0)