New Connector Destination: TimescaleDB

[MattMacGillivray]

Tell us about the new connector you’d like to have

• Sources: Postgres, Postgres with the Timescale extension (specifically hypertables), Azure Managed Postgres Databases with the Timescale extension
• Destinations: Postgres with the Timescale extension (specifically hypertables), managed / cloud timescale

Describe the context around this new connector

• Timescale is a timeseries extension for postgres that allows considerable performance improvements for timeseries data
• Timescale now offers cloud and managed databases, which offer functionality not available in the apache licensed Azure (and other) hosted postgres databases using the timescale extension (compression, continuous aggregates, etc).
• Currently, the only way to migrate from an existing Timescale database to the Timescale managed/cloud database is to do a full pgdump/restore , which is not ideal for larger databases. In my example, I have a few 3TB postgres/timesclae databases running as azure managed postgres databases. pgdump/restore would require a huge amount of downtime, or an overly complicated approach to writing data in parallel.
• Syncs would run continuously until a migration occurs.
• Syncs could also run daily for batch operations.
• Why is this hard? Because timescale's hypertables don't support logical replication , instead support postgres streaming replication.
• I am a mutual user of airbyte and timescaledb, you can reference InnerSpace.io .

Describe the alternative you are considering or using

I won't move to this other hosted database.

[marcosmarxm]

Thanks for requesting this @qmnonic. Do you know if TimescaleDB has a JDBC driver? Most destinations in Airbyte work with an Abstract using JDBC and if TimescaleDB has the support would be easier to implement this connector.

[MattMacGillivray]

TimescaleDB runs on top of postgres, and yes, it has JDBC connectivity - https://docs.timescale.com/timescaledb/latest/quick-start/java/#pre-requisites

…

Message ID: ***@***.***>

[aaronovz1]

Any movement on this one?

[kodeine]

Im interested in this as well

[timoconnellaus]

It working for me. Use these settings:

host: .tsdb.cloud.timescale.com
port: your port
dbname: your db (mine defaulted to tsdb)
username: your username
defatult schema: public is what I've used
optional fields -> password: your password
ssl mode: require