Assistance with error: $.date_of_transfer: does not match the date-time pattern must be a valid RFC 3339 date-time

[hskev]

Hi,

Getting the error in the title on import of a large CSV file that I can't seem to figure out if it's a bug or a problem with my configuration. This used to work in the pre 1.0 versions with the same file format.

Airbyte Community V1.4.0 (Via Helm)
Source connector: File (CSV) v0.5.18
Dest connector: MySQL v.1.0.3

File CSV reader options:

{
    "names": ["...", "date_of_transfer", "..."], 
    "parse_dates": ["date_of_transfer"], 
    "date_format": "%Y-%m-%d %H:%M"
}

The Connection schema identifies it as Datetime, which is correct.

The column in the source CSV has the following format: YYYY-mm-dd HH:MM (eg: "1995-11-17 00:00")

in the _airbyte_extracted_at table on the destination, I can see it parsed:
{"...", "date_of_transfer": "1995-11-17T00:00:00"}

However, in the output table, the column is null. All other columns are extracted properly.
The output table has the column as a VARCHAR(1024)

Error in logs:

2025-01-30 14:47:44 replication-orchestrator INFO Stream status TRACE received of status: COMPLETE for stream lrppd
2025-01-30 14:47:44 replication-orchestrator INFO Total records read: 13 (3 KB)
2025-01-30 14:47:44 replication-orchestrator INFO Schema validation was performed to a max of 10 records with errors per stream.
2025-01-30 14:47:44 replication-orchestrator WARN Schema validation errors found for stream _lrp. Error messages: [$.date_of_transfer: does not match the date-time pattern must be a valid RFC 3339 date-time]
2025-01-30 14:47:44 replication-orchestrator INFO readFromSource: done. (source.isFinished:true, fromSource.isClosed:false)
2025-01-30 14:47:44 replication-orchestrator INFO thread status... heartbeat thread: false , replication thread: true
2025-01-30 14:47:49 replication-orchestrator INFO processMessage: done. (fromSource.isDone:true, forDest.isClosed:false)
2025-01-30 14:47:49 replication-orchestrator INFO writeToDestination: done. (forDest.isDone:true, isDestRunning:true)
2025-01-30 14:47:49 replication-orchestrator INFO thread status... timeout thread: false , replication thread: true

Does anyone have any suggestions?

If you want a CSV file to test this with: https://www.gov.uk/government/statistical-data-sets/price-paid-data-downloads#december-2024-data-current-month

FULL Reader options:

{"names": ["tui", "price", "date_of_transfer", "postcode", "ptype", "new", "duration", "paon", "saon", "street", "locality","city","r_district","county", "ppdct", "status"], "parse_dates": ["date_of_transfer"], "date_format": "%Y-%m-%d %H:%M"}

Any help is greatly appreciated, thanks!