[destination-tidb, normalization] how to change varchar(1000) limit?

[yevhen-korientsov]

Hi,
we have a connection from Mongodb to Tidb, airbyte creates schema automatically, and it creates varchar(1000) data type in Tidb.
And we have some fields which are longer than 1000, so we faced error:
1406 (22001): Data Too Long, field len 1000, data len 1101

Is there some simple way to increase this value from 1000 ? (Tidb has limit 16383)

[marcosmarxm]

Today the destination TiDB uses Airbyte normalization, a deprecated features, to create the final tables.
You can check here where this happens. If you need to change that, you probably need to edit the normalization module and rebuild it from scratch to uses the new version.
Another way and more practical is to export the generated dbt project and use a custom transformation, instructions you can read here: https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-sql