passthroughTR

passthroughTR Transformer

Purpose

This transformer does ... nothing. However it couls be used when it's needed to change the Dataset name (id) or when no transformation is needed between one or several extraction and one or more loadings.

It works in a very simple way:

• It accepts many datasets in input.
• If one dataset is specified in input it must have a corresponding (same order) dataset in output. Said differently the configuration must have the same number of dataset in input and in output.

Configuration by using the SQL directly in the configuration file

The specific configuration (as a Datasource) in the configuration file section parameters does not includes specifics parameters.

Data sources configuration

• Inputs : several inputs
• Outputs : same number of outputs

Configuration example:

    "transformers":  [
    { 
        "id": "T1",
        "classname": "pipelite.transformers.passthroughTR",
        "inputs" : [ "S1", "S2" ],
        "outputs" : [ "O1", "O2" ]
    }
    ... ] ...

Example

graph TD;
    id1[Read Data Source S1]-->id2[Dataset S1];
    id2[Dataset S1]-->id3[passthroughTR S1 To S2];
    id3[passthroughTR S1 To S2]-->id4[Write Data source S2];

Loading

See here

In this example no transformation is performed: only a read (S1) and write (to another csv file: S2) is requested. The passthroughTR transfomer is used to indicates to the next loader (S2) it needs to be loaded. Without this transformer no loading would have been made and only the data source S1 would have been read.