Skip to content

KAFKA-19209: Clarify index.interval.bytes impact on offset and time index #686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: asf-site
Choose a base branch
from
Open

KAFKA-19209: Clarify index.interval.bytes impact on offset and time index #686

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f9b31ae
docs: Clarify that index.interval.bytes affects both offset and time …
shmily7829 Apr 29, 2025
46ea941
KAFKA-19209: Improve clarity and readability of index.interval.bytes …
shmily7829 May 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion 40/generated/topic_config.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,12 @@ <h4><a id="follower.replication.throttled.replicas"></a><a id="topicconfigs_foll
</li>
<li>
<h4><a id="index.interval.bytes"></a><a id="topicconfigs_index.interval.bytes" href="#topicconfigs_index.interval.bytes">index.interval.bytes</a></h4>
<p>This setting controls how frequently Kafka adds an index entry to its offset index. The default setting ensures that we index a message roughly every 4096 bytes. More indexing allows reads to jump closer to the exact position in the log but makes the index larger. You probably don't need to change this.</p>
<p>This setting controls how frequently Kafka adds entries to its offset index and, conditionally, to its time index. After appending a batch of messages, if the total bytes written since the last index entry exceed this value, Kafka will:</p>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @shmily7829, thanks for the PR!
Normally we'll fix the config doc in https://github.com/apache/kafka/blob/269e2d898b76b5c9c58232c35bee805ceacc2ead/clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java#L117

This doc you are editing here is generated by script.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @brandboat,
Thank you for the reminder.
I will make the corrections by this weekend.

Copy link
Author

@shmily7829 shmily7829 May 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updated PR has been submitted at apache/kafka#19657.
Therefore, I’ll go ahead and close this one.
@brandboat Thank you for your time and review!

<ul>
<li>Add an entry to the offset index for the batch's last offset.</li>
<li>Attempt to add an entry to the time index if the batch's maximum timestamp is greater than the last indexed timestamp.</li>
</ul>
<p>The default setting ensures that we index a message roughly every 4096 bytes. More frequent indexing allows reads to jump closer to the exact position in the log but results in larger index files. You probably don't need to change this.</p>
<table><tbody>
<tr><th>Type:</th><td>int</td></tr>
<tr><th>Default:</th><td>4096 (4 kibibytes)</td></tr>
Expand Down