dlt: data load tool

[amotl]

About

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

Details

dlt is an open-source Python library that loads data from various, often messy data sources into well-structured, live datasets. It offers a lightweight interface for extracting data from REST APIs, SQL databases, cloud storage, Python data structures, and many more.

Resources

• https://dlthub.com/docs/intro
• https://github.com/dlt-hub/dlt

Requests

• ingestr: CLI tool to copy data between any databases #86

[amotl]

We have been able to make some progress on unlocking dlt and ingestr with CrateDB.

• CrateDB: Add destination adapter dlt-hub/dlt#2733
• Support for CrateDB bruin-data/ingestr#238
• dlt: Hello, World! cratedb-examples#875

[amotl]

For reading data from CrateDB, dlt.sources.sql_database is the proper source element, using SQLAlchemy. Note to self: Submit a similar patch like crate/cratedb-examples#875 to demonstrate this.