Skip to content

kafka delay metric not reporting when count drops to zero #17974

@alexr17

Description

@alexr17

Bug Description

What happened:

The kafkaDelayCount metric stops being emitted when Kafka lag drops to zero because the metric call in KafkaOffsetGen.getNextOffsetRanges() is inside a checkpoint validation conditional block.

What you expected:

The metric should always be emitted, reporting 0 when caught up rather than disappearing/flatlining from monitoring dashboards.

Steps to reproduce:
Start DeltaStreamer with Kafka source and metrics enabled
Wait for consumer to catch up (latestOffsets == committedOffsets)
Observe kafkaDelayCount metric is no longer emitted

Environment

Hudi version:
Query engine: (Spark/Flink/Trino etc)
Relevant configs:

Logs and Stack Trace

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:bugBug reports and fixes

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions