The IBMi connector crashes due to unsupported BLOB data type #1777
Description
Bug report
What Debezium connector do you use and what version?
debezium-connector-ibmi:v3.5.0, but also in prior versions
What is the connector configuration?
column.exclude.list: ".*EXAMPLE"
What is the captured database version and mode of deployment?
E.g. on-premises, with a specific cloud provider, etc.
IBMi AS400 DB2
What behavior do you expect?
I would expect the connector to start even if the table contains a BLOB type column as long as this column is excluded
What behavior do you see?
Connector crashes with this error, where EXAMPLE obviously is not excluded.
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.\n\tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:67)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:155)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat io.debezium.util.Threads$3.lambda$newThread$0(Threads.java:314)\n\tat java.base/java.lang.Thread.run(Thread.java:840)\nCaused by: io.debezium.DebeziumException: java.lang.IllegalArgumentException: Unsupported type BLOB for column EXAMPLE\n\tat io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:111)\n\tat io.debezium.connector.db2as400.As400SnapshotChangeEventSource.execute(As400SnapshotChangeEventSource.java:70)\n\tat io.debezium.connector.db2as400.As400SnapshotChangeEventSource.execute(As400SnapshotChangeEventSource.java:38)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:311)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:295)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:206)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:148)\n\t... 6 more\nCaused by: java.lang.IllegalArgumentException: Unsupported type BLOB for column IDPHOTO\n\tat io.debezium.ibmi.db2.journal.retrieve.JdbcFileDecoder.toDataType(JdbcFileDecoder.java:351)\n\tat io.debezium.connector.db2as400.conversion.SchemaInfoConversion.table2As400Structure(SchemaInfoConversion.java:60)\n\tat io.debezium.connector.db2as400.conversion.SchemaInfoConversion.table2TableInfo(SchemaInfoConversion.java:37)\n\tat io.debezium.connector.db2as400.As400DatabaseSchema.lambda$addSchema$1(As400DatabaseSchema.java:79)\n\tat java.base/java.util.Optional.map(Optional.java:260)\n\tat io.debezium.connector.db2as400.As400DatabaseSchema.addSchema(As400DatabaseSchema.java:78)\n\tat io.debezium.connector.db2as400.As400SnapshotChangeEventSource.readTableStructure(As400SnapshotChangeEventSource.java:170)\n\tat io.debezium.connector.db2as400.As400SnapshotChangeEventSource.readTableStructure(As400SnapshotChangeEventSource.java:38)\n\tat io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:171)\n\tat io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:102)\n\t... 12 more\n
Do you see the same behaviour using the latest released Debezium version?
Ideally, also verify with the latest Alpha/Beta/CR version.
Yes
Do you have the connector logs, ideally from start till finish?
You might be asked later to provide DEBUG/TRACE level log.
see above
How to reproduce the issue using our tutorial deployment?
see above