Can Sling do the T part of ELT?

[alberttwong]

It doesn't quite say it in the docs but I'm curious.

[flarco]

Sling only does the following transformations:

• Column type coercion, the columns key in source.options (https://docs.slingdata.io/sling-cli/run/configuration#source)
• Special string transforms, like encodings, uuid parsing. See https://github.com/slingdata-io/sling-cli/blob/main/core/sling/tranforms.go
• JSON Flattenning: the flatten key in source.options (https://docs.slingdata.io/sling-cli/run/configuration#source)
• JMESPath transformations for JSON source files: the jmespath key in source.options (https://docs.slingdata.io/sling-cli/run/configuration#source)
• Custom SELECT query as the source stream.

There are no plans to supports other custom transformations.

[alberttwong]

We really should publish this info on the docs page.

[flarco]

Will do!

[flarco]

Added with examples 👉 https://docs.slingdata.io/sling-cli/run/transformations

[miguelgilmartinez]

Yes. I confirm I was able to export from a StarRocks table composed by 6 columns just the two of them to a MySQL blank schema. Sling created the new table and imported data properly.

./sling run --src-conn STARROCKS --src-stream "SELECT columnB, columnA FROM tbl WHERE columnB > 6000" -tgt-conn MYSQL --tgt-object=tbl --mode full-refresh

In that proof of concept, Sling executed the SELECT and created a new table with right data types. Great job.

Note: Probably it is not a proper "T" part by Sling, but it can do the job.

[flarco]

True, added to the list