diff --git a/docs/assets/read_batch.png b/docs/assets/read_batch.png new file mode 100644 index 0000000000..47c2de115f Binary files /dev/null and b/docs/assets/read_batch.png differ diff --git a/docs/user-guide/reference/configuration/read-batch-size.md b/docs/user-guide/reference/configuration/read-batch-size.md new file mode 100644 index 0000000000..de3a93fdd6 --- /dev/null +++ b/docs/user-guide/reference/configuration/read-batch-size.md @@ -0,0 +1,25 @@ +### Understanding `readBatchSize` + +At its core, `readBatchSize` is the **maximum number of messages a single vertex replica (pod) will try to read per cycle**. + + +#### **Default Behavior (Single Partition)** + +By default, the edge connecting vertices has one partition. In this common setup, a single replica reads from that single partition. + +**Note**: Monovertex always follows this default behavior, as it does not have the concept of partitions. +* **Example:** If `readBatchSize` is **500** and you have **1** replica, that replica will read up to **500** messages in each processing cycle. Simple! + +#### **Behavior with Multi-Partitioned Edges** + +When you configure a vertex to have multiple partitions (**not applicable for Source Vertices**), you increase its ability to process data in parallel. The key thing to understand is that the `readBatchSize` limit applies to the read operation on **each partition** that a replica is responsible for. + +This means the total number of messages a single replica can process per cycle becomes: +$$\text{Total Messages per Replica} = \text{readBatchSize} \times \text{Number of Partitions}$$ + +* **Example:** Let's say you configure a vertex to have **3** partitions and it's running with **1** replica. If `readBatchSize` is **500**: + * The single replica will read up to **500** messages from Partition 1. + * It will *also* read up to **500** messages from Partition 2 and 3. + * In total, that single replica can now process up to **1,500** messages per cycle (500 * 3). + +![read_batch.png](../../../assets/read_batch.png) diff --git a/docs/user-guide/reference/pipeline-tuning.md b/docs/user-guide/reference/pipeline-tuning.md index 8dcaaa2783..02751d1a44 100644 --- a/docs/user-guide/reference/pipeline-tuning.md +++ b/docs/user-guide/reference/pipeline-tuning.md @@ -5,15 +5,25 @@ In a data processing [pipeline](../../core-concepts/pipeline.md), certain parame ## Vertex Tuning Each [vertex](../../core-concepts/vertex.md) keeps running the cycle of reading data from an Inter-Step Buffer (or data source), -processing the data, and writing to the next Inter-Step Buffers (or sinks). There are some parameters can be adjusted for this data +processing the data, and writing to the next Inter-Step Buffers (or sinks). There are some parameters that can be adjusted for this data processing cycle. -- `readBatchSize` - The number of messages to read in each cycle, with a default value of `500`. It works together with `readTimeout` during a read operation, concluding when either limit is reached first. -- `readTimeout` - Read timeout from the source or Inter-Step Buffer, defaults to `1s`. It works in conjunction with `readBatchSize`. -- `bufferMaxLength` - How many unprocessed messages can be existing in the Inter-Step Buffer, defaults to `30000`. -- `bufferUsageLimit` - The percentage of the buffer usage limit, a valid number should be less than 100. Default value is `80`, which means `80%`. +- **`readBatchSize`** — Maximum number of messages **each vertex replica** reads **per cycle**. + *Default:* `500`. A cycle ends when either this cap is reached **or** `readTimeout` expires. + +**NOTE**: If you are working with [Multi-Partitioned edges](multi-partition.md), please refer to the advanced + [configuration](./configuration/read-batch-size.md) for `readBatchSize`. -These parameters can be customized under `spec.limits` as below, once defined, they apply to all the vertices and Inter-Step Buffers of the pipeline. +- **`readTimeout`** — Read timeout from the source or Inter-Step Buffer. + *Default:* `1s`. Works in conjunction with `readBatchSize` (whichever condition is met first ends the cycle). + +- **`bufferMaxLength`** — How many unprocessed messages can exist in the Inter-Step Buffer. + *Default:* `30000`. + +- **`bufferUsageLimit`** — The percentage threshold for buffer usage; must be `< 100`. + *Default:* `80` (i.e., `80%`). + +These parameters can be customized under `spec.limits` as below; once defined, they apply to all the vertices and Inter-Step Buffers of the pipeline. ```yaml apiVersion: numaflow.numaproj.io/v1alpha1 diff --git a/mkdocs.yml b/mkdocs.yml index 3e3f86726f..56c799f55d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -119,6 +119,7 @@ nav: - user-guide/reference/configuration/pipeline-customization.md - user-guide/reference/configuration/dra.md - user-guide/reference/configuration/istio.md + - user-guide/reference/configuration/read-batch-size.md - user-guide/reference/configuration/max-message-size.md - user-guide/reference/configuration/update-strategy.md - user-guide/reference/kustomize/kustomize.md