Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assets/read_batch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 25 additions & 0 deletions docs/user-guide/reference/configuration/read-batch-size.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
### Understanding `readBatchSize`

At its core, `readBatchSize` is the **maximum number of messages a single vertex replica (pod) will try to read per cycle**.


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about adding/linking some pointers regarding the parallelization with the new backend?
@vigith thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't change much here re: new backend.

#### **Default Behavior (Single Partition)**

By default, the edge connecting vertices has one partition. In this common setup, a single replica reads from that single partition.

**Note**: Monovertex always follows this default behavior, as it does not have the concept of partitions.
* **Example:** If `readBatchSize` is **500** and you have **1** replica, that replica will read up to **500** messages in each processing cycle. Simple!

#### **Behavior with Multi-Partitioned Edges**

When you configure a vertex to have multiple partitions (**not applicable for Source Vertices**), you increase its ability to process data in parallel. The key thing to understand is that the `readBatchSize` limit applies to the read operation on **each partition** that a replica is responsible for.

This means the total number of messages a single replica can process per cycle becomes:
$$\text{Total Messages per Replica} = \text{readBatchSize} \times \text{Number of Partitions}$$

* **Example:** Let's say you configure a vertex to have **3** partitions and it's running with **1** replica. If `readBatchSize` is **500**:
* The single replica will read up to **500** messages from Partition 1.
* It will *also* read up to **500** messages from Partition 2 and 3.
* In total, that single replica can now process up to **1,500** messages per cycle (500 * 3).

![read_batch.png](../../../assets/read_batch.png)
22 changes: 16 additions & 6 deletions docs/user-guide/reference/pipeline-tuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,25 @@ In a data processing [pipeline](../../core-concepts/pipeline.md), certain parame
## Vertex Tuning

Each [vertex](../../core-concepts/vertex.md) keeps running the cycle of reading data from an Inter-Step Buffer (or data source),
processing the data, and writing to the next Inter-Step Buffers (or sinks). There are some parameters can be adjusted for this data
processing the data, and writing to the next Inter-Step Buffers (or sinks). There are some parameters that can be adjusted for this data
processing cycle.

- `readBatchSize` - The number of messages to read in each cycle, with a default value of `500`. It works together with `readTimeout` during a read operation, concluding when either limit is reached first.
- `readTimeout` - Read timeout from the source or Inter-Step Buffer, defaults to `1s`. It works in conjunction with `readBatchSize`.
- `bufferMaxLength` - How many unprocessed messages can be existing in the Inter-Step Buffer, defaults to `30000`.
- `bufferUsageLimit` - The percentage of the buffer usage limit, a valid number should be less than 100. Default value is `80`, which means `80%`.
- **`readBatchSize`** — Maximum number of messages **each vertex replica** reads **per cycle**.
*Default:* `500`. A cycle ends when either this cap is reached **or** `readTimeout` expires.

**NOTE**: If you are working with [Multi-Partitioned edges](multi-partition.md), please refer to the advanced
[configuration](./configuration/read-batch-size.md) for `readBatchSize`.

These parameters can be customized under `spec.limits` as below, once defined, they apply to all the vertices and Inter-Step Buffers of the pipeline.
- **`readTimeout`** — Read timeout from the source or Inter-Step Buffer.
*Default:* `1s`. Works in conjunction with `readBatchSize` (whichever condition is met first ends the cycle).

- **`bufferMaxLength`** — How many unprocessed messages can exist in the Inter-Step Buffer.
*Default:* `30000`.

- **`bufferUsageLimit`** — The percentage threshold for buffer usage; must be `< 100`.
*Default:* `80` (i.e., `80%`).

These parameters can be customized under `spec.limits` as below; once defined, they apply to all the vertices and Inter-Step Buffers of the pipeline.

```yaml
apiVersion: numaflow.numaproj.io/v1alpha1
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ nav:
- user-guide/reference/configuration/pipeline-customization.md
- user-guide/reference/configuration/dra.md
- user-guide/reference/configuration/istio.md
- user-guide/reference/configuration/read-batch-size.md
- user-guide/reference/configuration/max-message-size.md
- user-guide/reference/configuration/update-strategy.md
- user-guide/reference/kustomize/kustomize.md
Expand Down
Loading