Skip to content

Commit 17e23e5

Browse files
authored
Apply suggestions from code review
1 parent 98a1c7c commit 17e23e5

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

modules/develop/pages/consume-data/consumer-offsets.adoc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,14 @@
44
// tag::single-source[]
55
:description: pass:q[Redpanda uses an internal topic, `__consumer_offsets`, to store committed offsets from each Kafka consumer that is attached to Redpanda.]
66

7-
== Overview
87

98
In Redpanda, all messages are organized by glossterm:topic[] and distributed across multiple partitions, based on a https://www.redpanda.com/guides/kafka-tutorial-kafka-partition-strategy[partition strategy^]. For example, when using the round robin strategy, a producer writing to a topic with five partitions would distribute approximately 20% of the messages to each glossterm:partition[].
109

1110
Within a partition, each message (once accepted and acknowledged by the partition leader) is permanently assigned a unique sequence number called an glossterm:offset[]. Offsets enable consumers to resume processing from a specific point, such as after an application outage. If an outage prevents your application from receiving events, you can use the consumer offset to retrieve only the events that occurred during the downtime. By default, the first message in a partition is assigned offset 0, the next is offset 1, and so on. You can manually specify a specific start value for offsets if needed. Once assigned, offsets are immutable, ensuring that the order of messages within a partition is preserved.
1211

12+
13+
== How consumers use offsets
14+
1315
As a consumer reads messages from Redpanda, it can save its progress by “committing the offset” (known as an glossterm:offset commit[]), an action initiated by the consumer, not Redpanda. Kafka client libraries provide an API for committing offsets, which communicates with Redpanda using the glossterm:consumer group[] API. Each committed offset is stored as a message in the `pass:[__consumer_offsets]` topic, which is a private Redpanda topic that stores committed offsets from each Kafka consumer attached to Redpanda, allowing the consumer to resume processing from the last committed point. Redpanda exposes the `pass:[__consumer_offsets]` key to enable the many tools in the Kafka ecosystem that rely on this value for their operation, providing greater ecosystem interoperability with environments and applications.
1416

1517
When a consumer group works together to consume data from topics, the partitions are divided among the consumers in the group. For example, if a topic has 12 partitions, and there are two consumers, each consumer would be assigned six partitions to consume. If a new consumer starts later and joins this consumer group, a rebalance occurs, such that each consumer ends up with four partitions to consume. You specify a consumer group by setting the `group.id` property to a unique name for the group.

0 commit comments

Comments
 (0)