Skip to content

Error deleting a row with NOT NULL bytea column #301

@hariso

Description

@hariso

Bug description

When deleting a row with a NOT NULL bytea column, the following error can be observed:

failed to encode before payload: failed to marshal data with schema: could not marshal into avro: col_bytea_not_null: avro: *avro.null is unsupported for Avro bytes"

Debugging this reveals that all the column values in the record.Payload.Before map are set to nil.

Steps to reproduce

  1. Create a source and destination table with a NOT NULL bytea column:
DROP TABLE IF EXISTS bytea_table_source;
CREATE TABLE bytea_table_source (
    id                      bigserial PRIMARY KEY,
    col_bytea_not_null      bytea NOT NULL
);

DROP TABLE IF EXISTS bytea_table_destination;
CREATE TABLE bytea_table_destination (
    id                      bigserial PRIMARY KEY,
    col_bytea_not_null      bytea NOT NULL
);
  1. Run the following pipeline:
version: "2.2"
pipelines:
  - id: pipeline1
    status: running
    name: pipeline1
    description: Postgres source, file destination
    connectors:
      - id: postgres-source
        type: source
        plugin: builtin:postgres
        name: source1
        settings:
          cdcMode: logrepl
          tables: bytea_table_source
          url: postgresql://meroxauser:meroxapass@localhost/meroxadb?sslmode=disable
      - id: pg-destination
        type: destination
        plugin: builtin:postgres
        name: pg-destination
        settings:
          url: postgresql://meroxauser:meroxapass@localhost/meroxadb?sslmode=disable
          table: bytea_table_destination
  1. Insert a row with: INSERT INTO bytea_table_source (col_bytea_not_null) VALUES ('aabbcc'::bytea);
  2. Delete with: delete from bytea_table_source;

Version

v0.14.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriageNeeds to be triaged

    Type

    No type

    Projects

    Status

    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions