Skip to content

JSON Schema verification - null value exceptions on non-required properties #843

@d4rkm4tterz

Description

@d4rkm4tterz

I'm having a weird problem when using the 10.6.x S3 connector (and also 10.5.7) with a JSON schema - whenever I add a non-required property I keep getting a "org.apache.kafka.connect.errors.DataException: Conversion error: null value for field that is required and has no default value" error when records are consumed by the connector.
So for example, using the following schema:

{
  "type":"object",
  "properties": {
    "customer_id" : {
      "type": "string",
      "default": ""
    },
    "correlation_id": {
      "type": "string"
    },
    "sku": {
      "type": "string"
    },
    "strategy": {
      "type": "integer"
    },
    "billing": {
      "type": "boolean"
    }
  },
  "additionalProperties": true
}

and this example event:

{
  "api_client_id": "",
  "auth_type": "",
  "billing": true,
  "correlation_id": "4f0144c2-5bd8-4ce2-b2c1-f18bfe674102",
  "country": "",
  "device_type": "",
  "erid": "",
  "event": "verify",
  "event_date": 0,
  "event_type": "not_applicable",
  "kyc": false,
  "msisdn": "",
  "pgcid": "998c1e0-173b-4a16-bcfb-182cc48fbb93-1737573592407",
  "sku": "sample_sku",
  "strategy": 1,
  "ts": 1745495247914
}

will result in the 'Conversion error' although the schema passes verification using 3rd party tools.

When I remove the customer_id prop from the schema, it works and the resulting event in S3 has only the props defined by the schema.

What's causing this behavior? Any suggestions welcome.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions