Skip to content

csvFileDS

datacorner edited this page Dec 1, 2023 · 2 revisions

Using the csvFileDS Data Source

Purpose

The aim of this Data Source is to read or write into a CSV file.

Pipelite Configuration

The specific configuration (as a Datasource) in the configuration file section parameters includes the following parameters:

  • separator: By default that's the comma (,) but the column separator can be changed here
  • filename: file name (without path)
  • path : path of the file to read and/or write

Configuration example:

"classname" : "pipelines.type.directPipeline",
"extractors" : [ {
    "id": "S1",
    "classname": "datasources.xesFileDS",
    "parameters": {
        "separator": ",",
        "filename": "test.xes",
        "path": "tests/data/"
    }   
    ... ] ...

Note: As the configuration of this Data source happens in the extractor section, pipelite will read the file. (it would have wrote into that file if the configuration would have happened in the loaders section).

🏠 Home
πŸ”‘ Main concepts
πŸ’» Installation
πŸ”¨ Configuration
πŸš€ Running

Supported Data Sources
πŸ“„ CSV File
πŸ“‘ XES File
πŸ“ƒ Excel File
πŸ“€ ODBC
🏒 SAP
🎒 ABBYY Timeline

Supported Transformations
πŸ”€ Pass Through
πŸ“Ά Dataset Profiling
πŸ”‚ Concat 2 Data sources
πŸ†– SubString
πŸ†’ Column Transformation
πŸ”ƒ Join data sources
πŸ”ƒ Lookup
πŸ”€ Rename Column Name

Extending pipelite
βœ… how to
βœ… Adding new Data sources
βœ… Adding new Transformers
βœ… Adding new Pipelines

Clone this wiki locally