Skip to content

Implement error handling mechanism for DataException #335

@devendra-nr

Description

@devendra-nr

Iceberg Kafka Connector fails when invalid message is received and records are not published to Iceberg or DLQ. To change this behaviour, support for following configurations for handling invalid data exception is implemented:

errors.tolerance: Behavior for tolerating errors during connector operation. 'none' is the default value and signals that any error will result in an immediate connector task failure; 'all' changes the behavior to skip over problematic records.

errors.log.include.messages: Whether to include in the log the Connect record that resulted in a failure. For sink records, the topic, partition, offset, and timestamp will be logged. For source records, the key and value (and their schemas), all headers, and the timestamp, Kafka topic, Kafka partition, source partition, and source offset will be logged. This is 'false' by default, which will prevent record keys, values, and headers from being written to log files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions