Getting RecordTooLargeException even after setting the configs to 100Mb

[alok87]

Error 1 seen

Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept.
2021-05-25 12:25:29,104 ERROR WorkerSourceTask{id=inventory-mysql-20210525-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-inventory-mysql-20210525-0]
2021-05-25 12:25:29,104 INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask) [task-thread-inventory-mysql-20210525-0]
2021-05-25 12:25:29,104 INFO Stopping MySQL connector task (io.debezium.connector.mysql.MySqlConnectorTask) [task-thread-inventory-mysql-20210525-0]
2021-05-25 12:25:29,105 INFO ChainedReader: Stopping the snapshot reader (io.debezium.connector.mysql.ChainedReader) [task-thread-inventory-mysql-20210525-0]
2021-05-25 12:25:29,107 INFO Discarding 8192 unsent record(s) due to the connector shutting down (io.debezium.connector.mysql.SnapshotReader) [task-thread-inventory-mysql-20210525-0]
...skipping...
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)

Error 2 seen

2021-05-25 12:25:29,119 INFO Step 9: rolling back transaction after abort (io.debezium.connector.mysql.SnapshotReader) [debezium-mysqlconnector-ts-snapshot]
2021-05-25 12:25:29,120 INFO Connector task finished all work and is now shutdown (io.debezium.connector.mysql.MySqlConnectorTask) [task-thread-inventory-mysql-20210525-0]
2021-05-25 12:25:29,120 INFO [Producer clientId=connector-producer-inventory-mysql-20210525-0] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer) [task-thread-inventory-mysql-20210525-0]
2021-05-25 12:25:29,143 ERROR Failed due to error: Aborting snapshot due to error when last running 'SELECT * FROM `luke`.`missing_drugs`': Can''t call rollback when autocommit=true (io.debezium.connector.mysql.SnapshotReader) [debezium-mysqlconnector-ts-snapshot]
org.apache.kafka.connect.errors.ConnectException: Can''t call rollback when autocommit=true Error code: 0; SQLSTATE: 08003.

Changes made in kafka connect resource is showing in the file:

sh-4.2$ cat /tmp/strimzi-connect.properties  | grep req
consumer.max.request.size=100000000
max.request.size=100000000
producer.max.request.size=100000000

Which error out of above 2 is causing the issue? Please suggest how to fix this.

[alok87]

Failure actually happen after this

2021-05-25 14:50:41,779 ERROR WorkerSourceTask{id=pharma-mysql-20210525-0} Failed to flush, timed out while waiting for producer to flush outstanding 3 messages (org.apache.kafka.connect.runtime.WorkerSourceTask) [task-thread-inventory-mysql-20210525-0]
2021-05-25 14:50:41,780 ERROR WorkerSourceTask{id=pharma-mysql-20210525-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-phar
ma-mysql-20210525-0]
org.apache.kafka.connect.errors.ConnectException: Unrecoverable exception from producer send callback

what to do?

[scholzj]

The maximum message size is configured in the broker. So you should make sure it is configured there. But 100MB messages are definitely not cool ... so sending smaller messages would be the right solution. I'm not sure about the other errors are they seem to be connector specific.

[alok87]

I was testing with 100Mb will it work, how to configure in the broker. I tried configuring max.request.size, producer.max.request.size and the consumer.max.request.size in the kafka connect resource. kubectl edit kc

[scholzj]

I do not know how to configure the details. Something might be covered in the docs or in the blog posts about consumer / producer tunning. Also Kafka documentation probably covers it.