Conversation
MarcialRosales
force-pushed
the
stream_offset_landmarks
branch
from
f73a0d6 to
9fa9c3f
Compare
MarcialRosales
force-pushed
the
stream_offset_landmarks
branch
from
16a9737 to
56d9665
Compare
- Reuse existing scaning functions - Specify -via argments- the exact offset percentiles (between 0.0 and 1.0). - Use samples rather than landmarks
Rather than using binary search and open/close on each seek attempt, instead we are going to determine which index files are going to be required and for each index file, which sample offsets we need to seek for. Then we use the existing (linear-search) function idx_skip_search with an accumulator function with the sample offsets served from the index file. If the 3 samples resides on a single index, it requires one open/close operation plus N reads, where N is proportional to the size of the index.
a) it leverages idx_skip_search b) it adds a callback function to idx_skip_search that is able to find several offsets in one scan
a) it leverages idx_skip_search b) it adds a callback function to idx_skip_search that is able to find several offsets in one scan
MarcialRosales
force-pushed
the
stream_offset_landmarks
branch
from
8a46b48 to
9a0ece8
Compare
kjnilsson
approved these changes
Mar 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces samples/2 on osiris_log: it walks the index and returns offset–timestamp samples for a list of fractions. Each fraction is in [0.0, 1.0]: 0.0 is the first message offset, 1.0 the last; values in between pick positions linearly along that range.
Used by the stream browser to render an offset bar with offset and timestamp at each landmark. The user can attach to a stream by dragging the control in the bar to the offsets they wish. Having landmarks helps the user jump to a specific offset or timestamp.