You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/develop/pages/consume-data/follower-fetching.adoc
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,9 +20,9 @@ The first fetch from a consumer is processed by a Redpanda leader broker. The le
20
20
21
21
Redpanda decides which replica a consumer fetches from. If the consumer configures its `client.rack` property, Redpanda by default selects a replica from the same rack as the consumer, if available.
22
22
23
+
ifndef::env-cloud[]
23
24
To enable follower fetching in Redpanda, configure properties for the consumer and the Redpanda cluster and broker:
24
25
25
-
ifndef::env-cloud[]
26
26
- For a Redpanda cluster, set the xref:reference:cluster-properties.adoc#enable_rack_awareness[`enable_rack_awareness`] property to `true`.
27
27
- For each Redpanda broker, set the xref:reference:node-properties.adoc#rack[`rack`] property to a rack ID.
28
28
- For each consumer, set the `client.rack` property to a rack ID.
:description: Learn about Leader Pinning and how to configure a preferred partition leader location based on cloud availability zones or regions.
3
+
:description: Learn about leader pinning and how to configure a preferred partition leader location based on cloud availability zones or regions.
4
+
5
+
Produce requests that write data to Redpanda topics go through the topic partition leader, which syncs messages across its follower replicas. For a Redpanda cluster deployed across multiple availability zones (AZs), leader pinning ensures that a topic's partition leaders are geographically closer to clients, which helps decrease networking costs and guarantees lower latency.
6
+
7
+
If consumers are located in the same preferred region or AZ for leader pinning, and you have not set up xref:develop:consume-data/follower-fetching.adoc[follower fetching], leader pinning can also help reduce networking costs on consume requests.
8
+
9
+
ifndef::env-cloud[]
10
+
== Prerequisites
4
11
5
12
[NOTE]
6
13
====
7
14
include::shared:partial$enterprise-license.adoc[]
8
15
====
9
16
10
-
Produce requests that write data to Redpanda topics go through the topic partition leader, which syncs messages across its follower replicas. For a Redpanda cluster deployed across multiple availability zones (AZs), Leader Pinning ensures that a topic's partition leaders are geographically closer to clients, which helps decrease networking costs and guarantees lower latency.
17
+
Before you can enable leader pinning, you must xref:manage:rack-awareness.adoc#configure-rack-awareness[configure rack awareness] on the cluster. If the config_ref:enable_rack_awareness,true,properties/cluster-properties[] cluster configuration property is set to `false`, leader pinning is disabled across the cluster.
11
18
12
-
If consumers are located in the same preferred region or AZ for Leader Pinning, and you have not set up xref:develop:consume-data/follower-fetching.adoc[follower fetching], Leader Pinning can also help reduce networking costs on consume requests.
19
+
endif::[]
13
20
14
-
== Prerequisites
21
+
ifndef::env-cloud[]
22
+
== Configure leader pinning
15
23
16
-
Before you can enable Leader Pinning, you must xref:manage:rack-awareness.adoc#configure-rack-awareness[configure rack awareness] on the cluster. If the config_ref:enable_rack_awareness,true,properties/cluster-properties[] cluster configuration property is set to `false`, Leader Pinning is disabled across the cluster.
17
-
18
-
== Configure Leader Pinning
19
-
20
-
You can use both a topic configuration property and a cluster configuration property to configure Leader Pinning.
21
-
22
-
You can set the topic configuration property for individual topics only, or set the cluster-wide configuration property that will enable Leader Pinning by default for all topics. You can also use a combination in which a default setting applies across the cluster, and you toggle the setting on or off for specific topics.
24
+
You can use both a topic configuration property and a cluster configuration property to configure leader pinning.
23
25
26
+
You can set the topic configuration property for individual topics only, or set the cluster-wide configuration property that will enable leader pinning by default for all topics. You can also use a combination in which a default setting applies across the cluster, and you toggle the setting on or off for specific topics.
24
27
25
28
This configuration is based on the following scenario: you have Redpanda deployed in a multi-AZ or multi-region cluster, and you have configured each broker so that the config_ref:rack,true,properties/broker-properties[] configuration property contains rack IDs corresponding to the AZ IDs:
26
29
27
30
* Set the topic configuration property xref:reference:properties/topic-properties.adoc#redpandaleaderspreference[`redpanda.leaders.preference`]. The property accepts the following string values:
28
31
+
29
32
--
30
-
** `none`: Opt out the topic from Leader Pinning.
33
+
**`none`: Opt out the topic from leader pinning.
31
34
**`racks:<rack1>[,<rack2>,...]`: Specify the preferred location (rack) of all topic partition leaders. The list can contain one or more rack IDs, and you can list the IDs in any order. Spaces in the list are ignored, for example: `racks:rack1,rack2` and `racks: rack1, rack2` are equivalent. You cannot specify empty rack IDs, for example: `racks: rack1,,rack2`. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.
32
35
--
33
36
+
34
37
This property inherits the default value from the cluster property `default_leaders_preference`.
35
38
36
-
* Set the cluster configuration property config_ref:default_leaders_preference,true,properties/cluster-properties[], which specifies the default Leader Pinning configuration for all topics that don’t have `redpanda.leaders.preference` explicitly set. It accepts values in the same format as `redpanda.leaders.preference`. Default: `none`
39
+
* Set the cluster configuration property config_ref:default_leaders_preference,true,properties/cluster-properties[], which specifies the default leader pinning configuration for all topics that don’t have `redpanda.leaders.preference` explicitly set. It accepts values in the same format as `redpanda.leaders.preference`. Default: `none`
37
40
+
38
41
This property also affects internal topics, such as `__consumer_offsets` and transaction coordinators. All offset tracking and transaction coordination requests get placed within the preferred regions or AZs for all clients, so you see end-to-end latency and networking cost benefits.
39
42
40
-
If there is more than one broker in the preferred AZ (or AZs), Leader Pinning distributes partition leaders uniformly across brokers in the AZ.
43
+
If there is more than one broker in the preferred AZ (or AZs), leader pinning distributes partition leaders uniformly across brokers in the AZ.
44
+
45
+
endif::[]
41
46
42
-
== Leader Pinning failover across availability zones
47
+
ifdef::env-cloud[]
48
+
== Configure leader pinning
49
+
50
+
Configure leader pinning if you have Redpanda deployed in a multi-AZ or multi-region cluster and your ingress is concentrated in a particular AZ or region.
51
+
52
+
Use the topic configuration property `redpanda.leaders.preference` to configure leader pinning for individual topics. The property accepts the following string values:
53
+
54
+
**`none`: Opt out the topic from leader pinning.
55
+
**`racks:<rack1>[,<rack2>,...]`: Specify the preferred location (rack) of all topic partition leaders. The list can contain one or more rack IDs, and you can list the IDs in any order. Spaces in the list are ignored, for example: `racks:rack1,rack2` and `racks: rack1, rack2` are equivalent. You cannot specify empty rack IDs, for example: `racks: rack1,,rack2`. If you specify multiple IDs, Redpanda tries to distribute the partition leader locations equally across brokers in these racks.
56
+
57
+
If there is more than one broker in the preferred AZ (or AZs), leader pinning distributes partition leaders uniformly across brokers in the AZ.
58
+
59
+
endif::[]
60
+
61
+
== Leader pinning failover across availability zones
43
62
44
63
If there are three AZs: A, B, and C, and A becomes unavailable, the failover behavior is as follows:
45
64
@@ -50,3 +69,5 @@ If there are three AZs: A, B, and C, and A becomes unavailable, the failover beh
0 commit comments