Uncaught exception in herder work thread

[chanvoyage]

After i add new plugin to kafka connect images:v2 and Deploy to existing kafka connect cluster During found error give me set up about 'cleanup.policy=compact' What did it mean?

kafka connect config yaml ================

========Error ============================

2021-09-27 11:32:13,803 WARN The configuration 'metrics.context.connect.group.id' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,803 WARN The configuration 'rest.advertised.host.name' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,803 WARN The configuration 'status.storage.topic' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 WARN The configuration 'client.rack' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 WARN The configuration 'rest.port' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 WARN The configuration 'config.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 WARN The configuration 'key.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 WARN The configuration 'value.converter.schemas.enable' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 WARN The configuration 'offset.storage.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 WARN The configuration 'admin.ssl.truststore.location' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 INFO Kafka version: 2.8.0 (org.apache.kafka.common.utils.AppInfoParser) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,804 INFO Kafka commitId: ebb1d6e21cc92130 (org.apache.kafka.common.utils.AppInfoParser) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,805 INFO Kafka startTimeMs: 1632717133804 (org.apache.kafka.common.utils.AppInfoParser) [DistributedHerder-connect-1-1]
2021-09-27 11:32:13,817 INFO Adding admin resources to main listener (org.apache.kafka.connect.runtime.rest.RestServer) [main]
2021-09-27 11:32:13,947 ERROR [Worker clientId=connect-1, groupId=kafka-connect-cluster] Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [DistributedHerder-connect-1-1]
org.apache.kafka.common.config.ConfigException: Topic 'kafka-connect-cluster-offsets' supplied via the 'offset.storage.topic' property is required to have 'cleanup.policy=compact' to guarantee consistency and durability of source connector offsets, but found the topic currently has 'cleanup.policy=delete'. Continuing would likely result in eventually losing source connector offsets and problems restarting this Connect cluster in the future. Change the 'offset.storage.topic' property in the Connect worker configurations to use a topic with 'cleanup.policy=compact'.
at org.apache.kafka.connect.util.TopicAdmin.verifyTopicCleanupPolicyOnlyCompact(TopicAdmin.java:526)
at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.lambda$createKafkaBasedLog$0(KafkaOffsetBackingStore.java:135)
at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:166)
at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:145)
at org.apache.kafka.connect.runtime.Worker.start(Worker.java:197)
at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:128)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:310)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2021-09-27 11:32:13,955 INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session) [main]
2021-09-27 11:32:13,955 INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session) [main]
2021-09-27 11:32:13,956 INFO node0 Scavenging every 600000ms (org.eclipse.jetty.server.session) [main]
2021-09-27 11:32:13,994 INFO Stopped http_8083@72eed547{HTTP/1.1, (http/1.1)}{0.0.0.0:8083} (org.eclipse.jetty.server.AbstractConnector) [JettyShutdownThread]
2021-09-27 11:32:13,997 INFO node0 Stopped scavenging (org.eclipse.jetty.server.session) [JettyShutdownThread]
2021-09-27 11:32:14,596 INFO Started o.e.j.s.ServletContextHandler@652b3733{/,null,SHUTDOWN} (org.eclipse.jetty.server.handler.ContextHandler) [main]
2021-09-27 11:32:14,597 INFO REST resources initialized; server is started and ready to handle requests (org.apache.kafka.connect.runtime.rest.RestServer) [main]
2021-09-27 11:32:14,597 INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect) [main]
2021-09-27 11:32:14,597 INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect) [connect-shutdown-hook]

[scholzj]

The error suggests that one of the Connect's internal topics has a wrong retention policy. You will need to edit the topic configuration and change it to compact - you should be able to do it either using the Topic Operator by editing the KafkaTopic resource or using the Kafka tools.