How to drop the two fields: "_airbyte_ab_id" and "_airbyte_emitted_at" from the output dump?

[hirads]

Hi,

I'm setting up a connection from Redshift to S3 but couldn't find any way to drop the two fields "_airbyte_ab_id" and "_airbyte_emitted_at" from the output. Anyone has an idea how to ignore them in the connection?

ps. I've read here that they are keeping track of important information, but I expected this to happen somewhere in the background and not on the final data dump.

[sajarin]

The fields "_airbyte_ab_id" and "_airbyte_emitted_at" are system fields added by Airbyte for data synchronization and tracking purposes. Currently, there is no built-in functionality to exclude these fields from the output.

However, you can create a transformation step after the extraction to remove these fields. This would involve creating a new table without these fields and copying the data over.

[meyerovb]

It would be faster (and maybe easier) to use unload or switch to spectrum depending on your use case