Resetting the last seen epoch of partition to 0 since the associated topicId changed from null

[ns-jhu]

I am using quay.io/strimzi/kafka:0.34.0-kafka-3.3.2 against a postgres database to generate CDC debezium messages. the postgres database had some wal file accumulated before.I checked the debezium, it works fine in the beginning:

2023-04-17 16:51:28,990 INFO [pg-introspect02-debezium-connector|task-0] 51266 records sent during previous 00:00:05.141, last recorded offset of {server=tpdata.pg-introspect02} partition is {last_snapshot_record=false, lsn=2125471940608, txId=110049, ts_usec=1681750284069020, snapshot=true} (io.debezium.connector.common.BaseSourceTask) [task-thread-pg-introspect02-debezium-connector-0]
2023-04-17 16:51:34,523 INFO [pg-introspect02-debezium-connector|task-0] 	 Exported 94854 records for table 'public.filemeta' after 00:00:10.287 (io.debezium.relational.RelationalSnapshotChangeEventSource) [pool-6-thread-1]
2023-04-17 16:51:39,095 INFO [pg-introspect02-debezium-connector|task-0] 65536 records sent during previous 00:00:10.105, last recorded offset of {server=tpdata.pg-introspect02} partition is {last_snapshot_record=false, lsn=2125471940608, txId=110049, ts_usec=1681750284069020, snapshot=true} (io.debezium.connector.common.BaseSourceTask) [task-thread-pg-introspect02-debezium-connector-0]
2023-04-17 16:51:44,613 INFO [pg-introspect02-debezium-connector|task-0] 	 Exported 153669 records for table 'public.filemeta' after 00:00:20.377 (io.debezium.relational.RelationalSnapshotChangeEventSource) [pool-6-thread-1]

but after running many hours, I see this

2023-04-18 15:17:47,881 INFO [pg-introspect02-debezium-connector|task-0] Snapshot ended with SnapshotResult [status=COMPLETED, offset=PostgresOffsetContext [sourceInfoSchema=Schema{io.debezium.connector.postgresql.Source:STRUCT}, sourceInfo=source_info[server='tpdata.pg-introspect02'db='app', lsn=LSN{1EE/E0000000}, txId=110049, timestamp=2023-04-17T16:51:24.069020Z, snapshot=FALSE, schema=public, table=filemeta], lastSnapshotRecord=true, lastCompletelyProcessedLsn=null, lastCommitLsn=null, streamingStoppingLsn=null, transactionContext=TransactionContext [currentTransactionId=null, perTableEventCount={}, totalEventCount=0], incrementalSnapshotContext=IncrementalSnapshotContext [windowOpened=false, chunkEndPosition=null, dataCollectionsToSnapshot=[], lastEventKeySent=null, maximumKey=null]]] (io.debezium.pipeline.ChangeEventSourceCoordinator) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:47,885 INFO [pg-introspect02-debezium-connector|task-0] Connected metrics set to 'true' (io.debezium.pipeline.ChangeEventSourceCoordinator) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:47,935 INFO [pg-introspect02-debezium-connector|task-0] REPLICA IDENTITY for 'public.filemeta' is 'FULL'; UPDATE AND DELETE events will contain the previous values of all the columns (io.debezium.connector.postgresql.PostgresSchema) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:47,938 INFO [pg-introspect02-debezium-connector|task-0] Starting streaming (io.debezium.pipeline.ChangeEventSourceCoordinator) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:47,938 INFO [pg-introspect02-debezium-connector|task-0] Retrieved latest position from stored offset 'LSN{1EE/E0000000}' (io.debezium.connector.postgresql.PostgresStreamingChangeEventSource) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:47,939 INFO [pg-introspect02-debezium-connector|task-0] Looking for WAL restart position for last commit LSN 'null' and last change LSN 'LSN{1EE/E0000000}' (io.debezium.connector.postgresql.connection.WalPositionLocator) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:47,939 INFO [pg-introspect02-debezium-connector|task-0] Initializing PgOutput logical decoder publication (io.debezium.connector.postgresql.connection.PostgresReplicationConnection) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:48,008 INFO [pg-introspect02-debezium-connector|task-0] Obtained valid replication slot ReplicationSlot [active=false, latestFlushedLsn=LSN{153/937AB8C8}, catalogXmin=69046] (io.debezium.connector.postgresql.connection.PostgresConnection) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:17:48,009 INFO [pg-introspect02-debezium-connector|task-0] Connection gracefully closed (io.debezium.jdbc.JdbcConnection) [pool-7-thread-1]
2023-04-18 15:17:48,010 INFO [pg-introspect02-debezium-connector|task-0] Seeking to LSN{1EE/E0000000} on the replication slot with command SELECT pg_replication_slot_advance('debezium', '1EE/E0000000') (io.debezium.connector.postgresql.connection.PostgresReplicationConnection) [debezium-postgresconnector-tpdata.pg-introspect02-change-event-source-coordinator]
2023-04-18 15:20:46,252 INFO [AdminClient clientId=adminclient-10] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 15:25:46,352 INFO [AdminClient clientId=adminclient-10] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 15:30:46,452 INFO [AdminClient clientId=adminclient-10] Node 2 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 15:35:46,552 INFO [AdminClient clientId=adminclient-10] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 15:40:46,656 INFO [AdminClient clientId=adminclient-10] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 15:45:46,760 INFO [AdminClient clientId=adminclient-10] Node 2 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 15:50:23,935 INFO [Producer clientId=producer-3] Resetting the last seen epoch of partition tpdata.pg-introspect02.connect-configs-0 to 0 since the associated topicId changed from null to D4C_N-xdQFGBEZPGYdXvYw (org.apache.kafka.clients.Metadata) [kafka-producer-network-thread | producer-3]
2023-04-18 15:50:23,941 INFO [Worker clientId=connect-1, groupId=tpdata-pg-introspect02-cluster] Session key updated (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [KafkaBasedLog Work Thread - tpdata.pg-introspect02.connect-configs]
2023-04-18 15:55:46,892 INFO [AdminClient clientId=adminclient-10] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:00:47,013 INFO [AdminClient clientId=adminclient-10] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:05:47,024 INFO [AdminClient clientId=adminclient-10] Node 2 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:10:47,140 INFO [AdminClient clientId=adminclient-10] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:15:47,268 INFO [AdminClient clientId=adminclient-10] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:20:47,368 INFO [AdminClient clientId=adminclient-10] Node 2 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:25:47,488 INFO [AdminClient clientId=adminclient-10] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:30:47,590 INFO [AdminClient clientId=adminclient-10] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:35:47,672 INFO [AdminClient clientId=adminclient-10] Node 2 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:40:47,780 INFO [AdminClient clientId=adminclient-10] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:45:47,800 INFO [AdminClient clientId=adminclient-10] Node 3 disconnected. (org.apache.kafka.clients.NetworkClient) [kafka-admin-client-thread | adminclient-10]
2023-04-18 16:50:23,937 INFO [Producer clientId=producer-3] Resetting the last seen epoch of partition tpdata.pg-introspect02.connect-configs-0 to 0 since the associated topicId changed from null to D4C_N-xdQFGBEZPGYdXvYw (org.apache.kafka.clients.Metadata) [kafka-producer-network-thread | producer-3]
2023-04-18 16:50:23,949 INFO [Worker clientId=connect-1, groupId=tpdata-pg-introspect02-cluster] Session key updated (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [KafkaBasedLog Work Thread - tpdata.pg-introspect02.connect-configs]

and no more messages can be produced from postgres table update.
my kafka server is 3.3.2 version.

once this "Resetting the last seen epoch of partition to 0" messages happens, nothing can help this situation any more , including restart kafka brokers, restart debezium. delete the topics and create them again. the only way to recover is to start from scratch, wipe out postgres database and create a new one, restart debezium. etc.

Does anybody know why this happens and what to do to work around this issue ?