Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 1 addition & 12 deletions modules/manage/pages/cluster-maintenance/cluster-balancing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -118,20 +118,9 @@ This mode is not recommended for production clusters. Only set to `off` if you n

== Intra-broker partition balancing

[NOTE]
====
This is a beta feature for v24.2 and is not recommended for use for production clusters. The feature is disabled by default. To unlock this feature, run this command with a superuser:

```bash
curl -X PUT -d '{"state": "active"}' http://127.0.0.1:9644/v1/features/node_local_core_assignment
```
====

In Redpanda, every partition replica is assigned to a CPU core on a broker. While Redpanda's default <<partition-replica-balancing,partition balancing>> monitors cluster-level events, such as the addition of new brokers or broker failure to balance partition assignments, it does not account for the distribution of partitions _within_ an individual broker.

Prior to Redpanda version 24.2, this meant that some cores on a broker could inadvertently host many partitions of heavily-used topics and cause the CPU to be xref:manage:monitoring.adoc#cpu-usage[overburdened]. Additionally, when the partition rebalance moved some partitions away from a broker, the remaining partitions did not necessarily rebalance across the broker's cores. Or, if a broker's core count was increased, Redpanda did not assign any partitions to the new cores until new partitions were created or old partitions were moved out.

Starting in v24.2, topic-aware intra-broker partition balancing allows for dynamically reassigning partitions within a broker. Redpanda prioritizes an even distribution of a topic's partition replicas across all cores in a broker. If a broker's core count changes, when the broker starts back up, Redpanda can check partition assignments across the broker's cores and reassign partitions, so that a balanced assignment is maintained across all cores. Redpanda can also check partition assignments when partitions are added to or removed from a broker, and rebalance the remaining partitions between cores.
Topic-aware intra-broker partition balancing allows for dynamically reassigning partitions within a broker. Redpanda prioritizes an even distribution of a topic's partition replicas across all cores in a broker. If a broker's core count changes, when the broker starts back up, Redpanda can check partition assignments across the broker's cores and reassign partitions, so that a balanced assignment is maintained across all cores. Redpanda can also check partition assignments when partitions are added to or removed from a broker, and rebalance the remaining partitions between cores.

To determine when to use intra-broker partition balancing, use the public metrics for CPU usage described in the xref:manage:monitoring.adoc#cpu-usage[Monitoring] guide, as well as the xref:reference:internal-metrics-reference.adoc#vectorized_reactor_utilization[`vectorized_reactor_utilization`] internal metric.

Expand Down
4 changes: 0 additions & 4 deletions modules/manage/pages/kubernetes/k-manage-resources.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@ kubectl describe nodes

- <<memory, Enable memory locking>>. This configuration prevents the operating system from paging out Redpanda's memory to disk, which can significantly impact performance.

== Limitations

Redpanda does not support decreasing the CPU cores for brokers in an existing cluster.

[[memory]]
== Configure memory resources

Expand Down