|
1 |
| -# csv-component |
| 1 | +# CSV Component |
2 | 2 |
|
3 |
| -A CSV component for the [elastic.io platform](http://www.elastic.io "elastic.io platform"). |
| 3 | +## Description |
4 | 4 |
|
| 5 | +A component to read and write Comma Separated Values (CSV) files. |
5 | 6 |
|
6 |
| -## Write CSV attachment |
| 7 | +## How works |
7 | 8 |
|
8 |
| -Multiple incoming events can be combined into one CSV file with the write CSV |
9 |
| -action. Incoming events will be written into the CSV file until there is a gap |
10 |
| -of more than 10 seconds between events. As part of the component setup, one |
11 |
| -must specify the columns of the CSV file. These columns will be published as |
12 |
| -the header in the first row. For each incoming event, the value for each header |
13 |
| -will be stringified and written as the value for that cell. All other |
14 |
| -properties will be ignored. For example, headers ``foo,bar`` along with the |
15 |
| -following JSON events |
| 9 | +The component can read the CSV file from a remote URL or from the message |
| 10 | +attachment. It can also write a CSV file from the incoming events. |
16 | 11 |
|
17 |
| - {"foo":"myfoo", "bar":"mybar"} |
18 |
| - {"foo":"myfoo", "bar":[1,2]} |
19 |
| - {"bar":"mybar", "baz":"mybaz"} |
| 12 | +## Requirements |
20 | 13 |
|
21 |
| -will produce the following ``.csv`` file: |
| 14 | +## Environment variables |
22 | 15 |
|
23 |
| - foo,bar |
24 |
| - myfoo,mybar |
25 |
| - myfoo,"[1,2]" |
26 |
| - ,mybar |
| 16 | +Component is not using any environment variables. |
27 | 17 |
|
28 |
| -When columns are added in the UI, you will be presented with an opportunity to |
29 |
| -provide a JSONata expression per column. If you require number formatting that |
30 |
| -is specific to a locale, the JSONata expression should handle that concern. |
31 | 18 |
|
32 |
| - |
| 19 | +## Credentials |
33 | 20 |
|
34 |
| -The output of the CSV Write component will be a message with an attachment. In |
35 |
| -order to access this attachment, the component following the CSV Write must be |
36 |
| -able to handle file attachments such as the [SFTP |
37 |
| -component](https://github.com/elasticio/sftp-component). |
| 21 | +The component does not require credentials to function. |
38 | 22 |
|
39 |
| -This is an open source component to work with |
40 |
| -[CSV](http://en.wikipedia.org/wiki/Comma-separated_values) files in your |
41 |
| -integration flow and is developed specifically to run on [elastic.io |
42 |
| -platform](http://www.elastic.io "elastic.io platform"). You can clone it and |
43 |
| -change it as you wish. However, **if you plan to deploy it into [elastic.io |
44 |
| -platform](http://www.elastic.io "elastic.io platform") you must follow sets of |
45 |
| -instructions to succeed**. |
46 | 23 |
|
47 |
| -> **PLEASE NOTE:** This is a working component for **CSV** files, however, it |
48 |
| -may not be suitable for every use case. Not all functionality is included in |
49 |
| -this component template. **It is your responsibility to add and modify this |
50 |
| -component to suit your integration needs.** |
| 24 | +## Triggers |
51 | 25 |
|
52 |
| -## Before you Begin |
| 26 | +### Read CSV file from URL |
53 | 27 |
|
54 |
| -Before you can deploy any code into our system **you must be a registered |
55 |
| -elastic.io platform user**. Please see our home page at |
56 |
| -[http://www.elastic.io](http://www.elastic.io) to learn how. |
| 28 | +This trigger will fetch the CSV file from a given URL. The address must be accessible |
| 29 | +to the component. The fetched CSV file will be placed in the attachment part of the |
| 30 | +outgoing message. |
57 | 31 |
|
58 |
| -> Any attempt to deploy a code into our platform without a registration would fail. |
| 32 | +## Actions |
59 | 33 |
|
60 |
| -After the registration and opening of the account you must **[upload your SSH |
61 |
| -Key](http://docs.elastic.io/docs/ssh-key)** into our platform. |
| 34 | +### Read CSV attachment |
62 | 35 |
|
63 |
| -> If you fail to upload you SSH Key you will get **permission denied** error |
64 |
| -during the deployment. |
| 36 | +This action will read the CSV attachment of the incoming message and output |
| 37 | +a `JSON` object. To configure this action the following fields can be used: |
65 | 38 |
|
66 |
| -## Getting Started |
| 39 | +* `CSV Header` - this is a required field. Input the names of headers separated with a comma. |
| 40 | +* `Separators` - Specify the separator type. Usually it is a comma (`,`) but values like Semicolon (`;`), Space (` `), Tab (`\t`) and Hash (`#`) are also supported. |
| 41 | +* `Skip rows` - if you know that the incoming CSV file has certain number of headers you can indicate to skip them. The supported values are `None`, `First row`, `First two`, `Dirst three` and `First four`. |
| 42 | +* `Data columns` - here the values will be added dynamically based on the values in the `CSV Header` field. Here each data column will be listed with the name, Data Type and the Format to enable further configuration. |
67 | 43 |
|
68 |
| -After registration and uploading of your SSH Key you can proceed to deploy it |
69 |
| -into our system. At this stage we suggest you to: |
70 |
| -* [Create a team](http://docs.elastic.io/page/team-management) to work on your |
71 |
| - new component. This is not required but will be automatically created using |
72 |
| - random naming by our system so we suggest you name your team accordingly. |
73 |
| -* [Create a repository](http://docs.elastic.io/page/repository-management) where |
74 |
| -* your new component is going to *reside* inside the team that you have just |
75 |
| -* created. For a simplicity you can name your repository **csv-component** or |
76 |
| -* **csv**. |
77 | 44 |
|
78 |
| -```bash |
79 |
| -$ git clone https://github.com/elasticio/csv-component.git csv-component |
| 45 | +### Write CSV attachment |
80 | 46 |
|
81 |
| -$ cd csv-component |
82 |
| -``` |
83 |
| -Now you can edit your version of **csv-component** component and change |
84 |
| -according to your needs - that is if you know what you are doing. Or you can |
85 |
| -just ``PUSH``it into our system to see the process in action: |
| 47 | +This action will combine multiple incoming events into a CSV file until there is a gap |
| 48 | +of more than 10 seconds between events. Afterwards, the CSV file will be closed |
| 49 | +and attached to the outgoing message. |
| 50 | + |
| 51 | +As part of the component setup, one must specify the columns of the CSV file. |
| 52 | +These columns will be published as the header in the first row. For each incoming |
| 53 | +event, the value for each header will be `stringified` and written as the value |
| 54 | +for that cell. All other properties will be ignored. For example, headers |
| 55 | +`foo,bar` along with the following JSON events: |
86 | 56 |
|
87 |
| -```bash |
88 |
| -$ git remote add elasticio [email protected]:csv-component.git |
| 57 | +``` |
| 58 | +{"foo":"myfoo", "bar":"mybar"} |
| 59 | +{"foo":"myfoo", "bar":[1,2]} |
| 60 | +{"bar":"mybar", "baz":"mybaz"} |
| 61 | +``` |
89 | 62 |
|
90 |
| -$ git push elasticio master |
| 63 | +will produce the following `.csv` file: |
91 | 64 | ```
|
92 |
| -Obviously the naming of your team and repository is entirely up-to you and if |
93 |
| -you do not put any corresponding naming our system will auto generate it for you |
94 |
| -but the naming might not entirely correspond to your project requirements. |
| 65 | +foo,bar |
| 66 | +myfoo,mybar |
| 67 | +myfoo,"[1,2]" |
| 68 | +,mybar |
| 69 | +``` |
| 70 | + |
| 71 | +When columns are added in the UI, you will be presented with an opportunity to |
| 72 | +provide a JSONata expression per column. If you require number formatting that |
| 73 | +is specific to a locale, the JSONata expression should handle that concern. |
| 74 | + |
| 75 | + |
| 76 | + |
| 77 | +The output of the CSV Write component will be a message with an attachment. In |
| 78 | +order to access this attachment, the component following the CSV Write must be |
| 79 | +able to handle file attachments. |
0 commit comments