-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
In what version(s) of Spring for Apache Kafka are you seeing this issue?
3.3.0
Describe the bug
Calling EmbeddedKafkaBroker#consumeFromAnEmbeddedTopic with seekToEnd set to true may cause the consumer to miss messages that were sent to the topic after calling that method, because the seeking is done asynchronously.
That seems unexpected and leads to flaky, unpredictable test results.
That is because consumeFromEmbeddedTopics calls the kafka Consumer#seekToEnd method which is documented as "This function evaluates lazily, seeking to the final offset in all partitions only when poll(Duration) or position(TopicPartition) are called.".
To Reproduce
Call consumeFromAnEmbeddedTopic with seekToEnd, then send a message to the topic, and then try to consume it. If the consumer sees the message or not depends on timing.
Concrete use case: Calling consumeFromAnEmbeddedTopic at the start of a test, then calling application code that is expected to send the message, and then using KafkaTestUtils.getSingleRecord in the test may randomly fail.
Workarounds: Calling Thread.sleep or Consumer.position after calling consumeFromAnEmbeddedTopic.
Expected behavior
The method should seek to the end before returning.
Sample
I will submit a PR including a failing test case.