Skip to content

fix the issue with offsetOutOfRange#485

Open
dethlex wants to merge 1 commit intolovoo:masterfrom
dethlex:fix-outside-offsets
Open

fix the issue with offsetOutOfRange#485
dethlex wants to merge 1 commit intolovoo:masterfrom
dethlex:fix-outside-offsets

Conversation

@dethlex
Copy link

@dethlex dethlex commented Mar 10, 2026

Summary
Handle ErrOffsetOutOfRange in goka's partition table recovery by falling back to sarama.OffsetOldest instead of crashing the processor

Problem

  Production error:                                                                                                                                                                                                                                                        
  error consuming %TOPIC%: kafka server: The requested offset is outside   
  the range of offsets maintained by the server for the given topic/partition         

Goka's findOffsetToLoad() adjusts offsets below the broker's oldest, but a race condition exists: the oldest offset can shift between the metadata fetch and the actual ConsumePartition call (e.g. due to log compaction or retention cleanup), causing ErrOffsetOutOfRange.

Fix When ConsumePartition returns ErrOffsetOutOfRange, retry with sarama.OffsetOldest (lets the broker resolve the actual earliest offset) instead of failing the partition setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant