Kafka Connector tasks are not failed but consumer leaves group and task is doing nothing

[cstmgl]

Hi all,

I'm not sure where to put this but I have a general question about the KafkaConnector that I'm struggling to understand.

I'm experiencing that my tasks of the connector are showing as running but the consumer actually leaves the group so actually nothing gets done.
Is this a default behavior? Is there a way to force the tasks to go into error status?

For example I see this in my log:

[Consumer clientId=connector-consumer-sink-jdbc-bde-objects-0, groupId=connect-sink-jdbc-bde-objects] Member connector-consumer-sink-jdbc-bde-objects-0-3333ce54-ddc4-41ad-878e-cc74f36e8ffd sending LeaveGroup request to coordinator abc-strimzi-kafka-kafka-0.abc-strimzi-kafka-kafka-brokers.insight-data-flow-nightly.svc:9093 (id: 2147483647 rack: null) due to consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records.

I understand there is connectivity issues or maybe too much load but I don't get way this does not raise an error in the task.
If I connect to the Kafka CLI I just see that there is no valid consumer-id for my connector group.

Is there any good documentation I can read that maybe can explain me this?
Or will this behavior be different per connector type and is not something generic in terms of Kafka Connectors?

[scholzj]

The error says that your consumer (= Connect & Connector) is not polling often enough. But I do not know what the reason would be.

[cstmgl]

I understood the issue, my point was if the task has no active consumers should it not go into error status?
This part of the behavior is what I not understand.
But I guess this really depends from connector to connector so I'll try to read on the connector side what this could be.

[cstmgl]

I guess the point here is that this is more dependent on the actual connectors and not the strimzi operations on the connectors.
I'm still not sure if this is a point about all connectors or this individual connectors but I'll try to ask elsewhere.