Skip to content

Commit 82c9046

Browse files
jhorbulykjhorbulyk
authored
Merge pull request #6 from elasticio/write-attachment-jhorbulyk
Add Write CSV Functionality
2 parents c33a250 + 1ef7d4b commit 82c9046

24 files changed

+2856
-593
lines changed

‎.eslintrc.js‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
'use strict';
2+
3+
const OFF = 'off';
4+
const ERROR = 'error';
5+
const WARN = 'warn';
6+
const ALWAYS = 'always';
7+
const NEVER = 'never';
8+
9+
module.exports = {
10+
'parserOptions': {
11+
ecmaVersion: 2017
12+
},
13+
'env': {
14+
es6: true,
15+
node: true,
16+
mocha: true
17+
},
18+
'plugins': [
19+
'mocha'
20+
],
21+
'extends': 'eslint:recommended',
22+
'rules': {
23+
'indent': [
24+
ERROR,
25+
4,
26+
{
27+
SwitchCase: 1
28+
}
29+
],
30+
'linebreak-style': ERROR,
31+
'quotes': [
32+
WARN,
33+
'single',
34+
{
35+
avoidEscape: true,
36+
allowTemplateLiterals: true
37+
}
38+
],
39+
'semi': [
40+
ERROR,
41+
ALWAYS
42+
],
43+
'func-names': ERROR,
44+
'no-var': ERROR,
45+
'no-empty': ERROR,
46+
'no-empty-function': ERROR,
47+
'brace-style': [
48+
ERROR,
49+
'1tbs',
50+
{ allowSingleLine: true }
51+
],
52+
'no-multiple-empty-lines': ERROR,
53+
'no-multi-spaces': ERROR,
54+
'one-var': [
55+
ERROR,
56+
NEVER
57+
],
58+
'quote-props': [
59+
WARN,
60+
'consistent-as-needed'
61+
],
62+
'key-spacing': ERROR,
63+
'space-unary-ops': [
64+
ERROR,
65+
{
66+
words: true,
67+
nonwords: false
68+
}
69+
],
70+
'no-spaced-func': ERROR,
71+
'space-before-function-paren': [
72+
ERROR,
73+
{
74+
anonymous: ALWAYS,
75+
named: NEVER
76+
}
77+
],
78+
'arrow-body-style': [
79+
WARN,
80+
'as-needed'
81+
],
82+
'array-bracket-spacing': ERROR,
83+
'space-in-parens': ERROR,
84+
'comma-dangle': ERROR,
85+
'no-trailing-spaces': ERROR,
86+
'yoda': ERROR,
87+
'max-len': [
88+
ERROR,
89+
120
90+
],
91+
'camelcase': [
92+
ERROR,
93+
{
94+
properties: 'never'
95+
}
96+
],
97+
'new-cap': [
98+
WARN,
99+
{
100+
capIsNewExceptions: ['Q']
101+
}
102+
],
103+
'comma-style': ERROR,
104+
'curly': ERROR,
105+
'object-curly-spacing': [
106+
WARN,
107+
ALWAYS
108+
],
109+
'object-curly-newline': [
110+
ERROR,
111+
{
112+
ObjectExpression: {
113+
minProperties: 1
114+
},
115+
ObjectPattern: {
116+
multiline: true,
117+
minProperties: 5
118+
}
119+
}
120+
],
121+
'object-property-newline': ERROR,
122+
'template-curly-spacing': ERROR,
123+
'dot-notation': ERROR,
124+
'dot-location': [
125+
ERROR,
126+
'property'
127+
],
128+
'func-style': [
129+
ERROR,
130+
'declaration',
131+
{
132+
allowArrowFunctions: true
133+
}
134+
],
135+
'eol-last': ERROR,
136+
'space-infix-ops': ERROR,
137+
'keyword-spacing': ERROR,
138+
'space-before-blocks': ERROR,
139+
'no-invalid-this': ERROR,
140+
'consistent-this': ERROR,
141+
'no-this-before-super': ERROR,
142+
'no-unreachable': ERROR,
143+
'no-sparse-arrays': ERROR,
144+
'array-callback-return': ERROR,
145+
'strict': [
146+
WARN,
147+
'global'
148+
],
149+
'eqeqeq': ERROR,
150+
'no-use-before-define': WARN,
151+
'no-undef': ERROR,
152+
'no-unused-vars': WARN,
153+
'no-mixed-spaces-and-tabs': ERROR,
154+
'operator-linebreak': [
155+
ERROR,
156+
'before'
157+
],
158+
'no-console': [
159+
OFF
160+
],
161+
"mocha/no-exclusive-tests": "error"
162+
}
163+
};

‎.jscsrc‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎.jshintrc‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎.travis.yml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
language: node_js
22
node_js:
33
- 8
4-
- 7
5-
- 6
4+
language: node_js
5+
node_js:
6+
- v8
7+
script: npm test && npm run integration-test
8+
notifications:
9+
email:
10+
- [email protected]

‎README.md‎

Lines changed: 57 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,36 +2,83 @@
22

33
> A CSV component for the [elastic.io platform](http://www.elastic.io "elastic.io platform").
44
5-
This is an open source component to work with [CSV](http://en.wikipedia.org/wiki/Comma-separated_values) files in your integration flow and is developed specifically to run on [elastic.io platform](http://www.elastic.io "elastic.io platform"). You can clone it and change it as you wish. However, **if you plan to deploy it into [elastic.io platform](http://www.elastic.io "elastic.io platform") you must follow sets of instructions to succeed**.
5+
This is an open source component to work with
6+
[CSV](http://en.wikipedia.org/wiki/Comma-separated_values) files in your
7+
integration flow and is developed specifically to run on [elastic.io
8+
platform](http://www.elastic.io "elastic.io platform"). You can clone it and
9+
change it as you wish. However, **if you plan to deploy it into [elastic.io
10+
platform](http://www.elastic.io "elastic.io platform") you must follow sets of
11+
instructions to succeed**.
612

7-
> **PLEASE NOTE:** This is a working component for **CSV** files, however, it may not be suitable for every use case. Not all functionalities are included in this component template. **It is your responsibility to add and modify this component to suit your integration needs.**
13+
> **PLEASE NOTE:** This is a working component for **CSV** files, however, it
14+
may not be suitable for every use case. Not all functionality is included in
15+
this component template. **It is your responsibility to add and modify this
16+
component to suit your integration needs.**
817

918
## Before you Begin
1019

11-
Before you can deploy any code into our system **you must be a registered elastic.io platform user**. Please see our home page at [http://www.elastic.io](http://www.elastic.io) to learn how.
20+
Before you can deploy any code into our system **you must be a registered
21+
elastic.io platform user**. Please see our home page at
22+
[http://www.elastic.io](http://www.elastic.io) to learn how.
1223

1324
> Any attempt to deploy a code into our platform without a registration would fail.
1425
15-
After the registration and opening of the account you must **[upload your SSH Key](http://docs.elastic.io/docs/ssh-key)** into our platform.
26+
After the registration and opening of the account you must **[upload your SSH
27+
Key](http://docs.elastic.io/docs/ssh-key)** into our platform.
1628

17-
> If you fail to upload you SSH Key you will get **permission denied** error during the deployment.
29+
> If you fail to upload you SSH Key you will get **permission denied** error
30+
during the deployment.
1831

1932
## Getting Started
2033

21-
After registration and uploading of your SSH Key you can proceed to deploy it into our system. At this stage we suggest you to:
22-
* [Create a team](http://docs.elastic.io/page/team-management) to work on your new component. This is not required but will be automatically created using random naming by our system so we suggest you name your team accordingly.
23-
* [Create a repository](http://docs.elastic.io/page/repository-management) where your new component is going to *reside* inside the team that you have just created. For a simplicity you can name your repository **csv-component** or **csv**.
34+
After registration and uploading of your SSH Key you can proceed to deploy it
35+
into our system. At this stage we suggest you to:
36+
* [Create a team](http://docs.elastic.io/page/team-management) to work on your
37+
new component. This is not required but will be automatically created using
38+
random naming by our system so we suggest you name your team accordingly.
39+
* [Create a repository](http://docs.elastic.io/page/repository-management) where
40+
* your new component is going to *reside* inside the team that you have just
41+
* created. For a simplicity you can name your repository **csv-component** or
42+
* **csv**.
2443

2544
```bash
2645
$ git clone https://github.com/elasticio/csv-component.git csv-component
2746

2847
$ cd csv-component
2948
```
30-
Now you can edit your version of **csv-component** component and change according to your needs - that is if you know what you are doing. Or you can just ``PUSH``it into our system to see the process in action:
49+
Now you can edit your version of **csv-component** component and change
50+
according to your needs - that is if you know what you are doing. Or you can
51+
just ``PUSH``it into our system to see the process in action:
3152

3253
```bash
3354
$ git remote add elasticio [email protected]:csv-component.git
3455

3556
$ git push elasticio master
3657
```
37-
Obviously the naming of your team and repository is entirely up-to you and if you do not put any corresponding naming our system will auto generate it for you but the naming might not entirely correspond to your project requirements.
58+
Obviously the naming of your team and repository is entirely up-to you and if
59+
you do not put any corresponding naming our system will auto generate it for you
60+
but the naming might not entirely correspond to your project requirements.
61+
62+
## Write CSV attachment
63+
64+
Multiple incoming events can be combined into one CSV file with the write CSV
65+
action. Incoming events will be written into the CSV file until there is a gap
66+
of more than 10 seconds between events. As part of the component setup, one
67+
must specify the columns of the CSV file. These columns will be published as
68+
the header in the first row. For each incoming event, the value for each header
69+
will be stringified and written as the value for that cell. All other
70+
properties will be ignored. For example, headers ``foo,bar`` along with the
71+
following JSON events
72+
73+
{"foo":"myfoo", "bar":"mybar"}
74+
{"foo":"myfoo", "bar":[1,2]}
75+
{"bar":"mybar", "baz":"mybaz"}
76+
77+
will produce the following ``.csv`` file:
78+
79+
foo,bar
80+
myfoo,mybar
81+
myfoo,"[1,2]"
82+
,mybar
83+
84+
Currently no formatting is performed as part of CSV Write.

‎component.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,21 @@
3737
"metadata": {
3838
"out": {}
3939
}
40+
},
41+
"write_attachment": {
42+
"description":
43+
"Multiple incoming events can be combined into one CSV file with the write CSV action. See https://github.com/elasticio/csv-component/ for additional documentation.",
44+
"main": "./lib/actions/write.js",
45+
"title": "Write CSV attachment",
46+
"fields": {
47+
"writer": {
48+
"viewClass": "CSVReadView"
49+
}
50+
},
51+
"metadata": {
52+
"in": {},
53+
"out": {}
54+
}
4055
}
4156
}
4257
}

‎gulpfile.js‎

Lines changed: 0 additions & 26 deletions
This file was deleted.

0 commit comments

Comments
 (0)