Postgres to Snowflake Transformations not supported?

[dfi69]

Hello everyone,
We're testing Airbyte to potentially unseat Fivetran in our organization. We've setup our main connector as Postgres -> Snowflake and replication is working great, except for 1 thing.
We're not able to apply any transformations on the connection, we're presented with the following message:

Normalization and Transformation operations are not supported for this connection.

Can someone shed some light on why we're facing this and what we would need to do to be able to use transformations.

Platform: 0.59.0 OSS
Source: 3.4.0
Destination: 3.7.4

Thanks,

[joeybenamy]

Normalization and custom transformation in Airbyte OSS are deprecated effective March 31. See #34860.

[dfi69]

@joeybenamy is this feature available in their commercial offerings or is it gone completely from their product?

[joeybenamy]

@joeybenamy is this feature available in their commercial offerings or is it gone completely from their product?

As far as I know, it is deprecated from all variations of their product. It is definitely gone from Airbyte OSS. The docs imply that it is also unavailable in cloud, enterprise, etc. in favor of typing and deduping in v2 destinations

[dfi69]

Thank you for the information @joeybenamy , I appreciate it!

[marcosmarxm]

Yes, custom transformation and normalization were deprecated. If you need to use post-transformations to your data Airbyte recommend to use dbt with an orchestrator to run the job to your pipeline.

[joeybenamy]

@dfi69 We replaced Fivetran with Airbyte for ETL from Postgres and MySQL to Snowflake. We are doing post-transformations with DBT running in our Kubernetes cluster that also runs Airbyte OSS. This works well for us; we never tried to use the normalization and custom normalization in Airbyte. We just use their typing and deduping. Kubernetes is effectively our orchestrator for DBT, for now.