Skip to content

Commit 0e18d5f

Browse files
Feediver1Deflaimun
andauthored
Caveat about adding partitions to existing topics (#891)
Co-authored-by: Paulo Borges <[email protected]>
1 parent 8993021 commit 0e18d5f

File tree

3 files changed

+11
-7
lines changed

3 files changed

+11
-7
lines changed

modules/develop/pages/config-topics.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,8 @@ rpk topic add-partitions [TOPICS...] --num [#]
7373

7474
Note that `--num <#>` is the number of partitions to _add_, not the total number of partitions.
7575

76+
include::develop:partial$balance-existing-topic-redistribution.adoc[]
77+
7678
ifndef::env-cloud[]
7779
=== Change the replication factor
7880

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
NOTE: If a topic already has messages and you add partitions, the existing messages won’t be redistributed to the new partitions. If you require messages to be redistributed, then you must create a new topic with the new partition count, then stream the messages from the old topic to the new topic so they are appropriately distributed according to the new partition hashing.

modules/manage/pages/cluster-maintenance/cluster-balancing.adoc

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
:page-aliases: cluster-administration:cluster-balancing.adoc
44
:page-categories: Management
55

6-
Cluster balancing is crucial for optimal performance. Unbalanced clusters can saturate resources on one or more brokers, impacting throughput and latency. Furthermore, a cluster with replicas on a down broker risks availability loss if more brokers fail, and a cluster that keeps losing brokers without healing eventually risks data loss. Redpanda provides various topic-aware tools to balance clusters for best performance.
6+
Cluster balancing is crucial for optimal performance. Unbalanced clusters can saturate resources on one or more brokers, impacting throughput and latency. Furthermore, a cluster with replicas on a down broker risks availability loss if more brokers fail, and a cluster that keeps losing brokers without healing eventually risks data loss. Redpanda provides various topic-aware tools to balance clusters for best performance.
77

88
[.no-clip]
99
|===
@@ -12,32 +12,33 @@ Cluster balancing is crucial for optimal performance. Unbalanced clusters can sa
1212
| <<Partition leadership balancing>>
1313
|
1414
This balancer transfers the leadership of a broker's partitions to other replicas to avoid topic leadership hotspots on one or a few specific brokers in your cluster.
15-
+
15+
1616
The partition leader regularly sends heartbeats to its followers. If a follower does not receive a heartbeat within a timeout, it triggers a new leader election. Redpanda also provides leadership balancing when brokers are added or decommissioned.
1717

1818
| <<Partition replica balancing>>
1919
|
2020
This balancer moves partition replicas to avoid topic replica hotspots on one or a few specific brokers in your cluster.
21-
+
21+
2222
Redpanda prioritizes balancing a topic's partition replica count evenly across all brokers while it's balancing the cluster's overall partition count. Because different topics in a cluster can have vastly different load profiles, this better distributes the workload evenly across brokers.
23-
+
23+
2424
Redpanda provides partition replica balancing when brokers are added or decommissioned.
2525

2626
| <<Intra-broker partition balancing>>
2727

2828
|
2929
This balancer moves partition replicas across CPU cores in an individual broker. Redpanda maintains balanced partition replica assignments between cores to avoid topic hotspots on one or a few specific cores within a broker.
30-
+
30+
3131
Continuous Intra-Broker Partition Balancing (`core_balancing_continuous`) requires an xref:get-started:licenses.adoc#redpanda-enterprise-edition[enterprise license].
3232

3333
| xref:manage:cluster-maintenance/continuous-data-balancing.adoc[Continuous Data Balancing]
3434

3535
|
36-
This balancer monitors broker and rack availability, as well as disk usage, to avoid topic hotspots when moving data off brokers with fuller disks. Continuous Data Balancing enables self-healing clusters that dynamically balance partitions. It also ensures adherence to rack-aware replica placement policy and self-heals after rack (or availability zone) failure or replacement. This balancer does not keep the relative fullness of each broker within a defined range, it just prevents hitting the fullness threshold of each individual broker.
37-
+
36+
This balancer monitors broker and rack availability, as well as disk usage, to avoid topic hotspots when moving data off brokers with fuller disks. Continuous Data Balancing enables self-healing clusters that dynamically balance partitions. It also ensures adherence to rack-aware replica placement policy and self-heals after rack (or availability zone) failure or replacement. This balancer does not keep the relative fullness of each broker within a defined range, it just prevents hitting the fullness threshold of each individual broker.
37+
3838
Continuous Data Balancing requires an xref:get-started:licenses.adoc#redpanda-enterprise-edition[enterprise license].
3939
|===
4040

41+
include::develop:partial$balance-existing-topic-redistribution.adoc[]
4142

4243
== Partition leadership balancing
4344

0 commit comments

Comments
 (0)