Skip to content
This repository was archived by the owner on Aug 12, 2022. It is now read-only.

Commit da00dd6

Browse files
committed
Update README.md
1 parent 26ed83c commit da00dd6

File tree

1 file changed

+44
-10
lines changed

1 file changed

+44
-10
lines changed

README.md

Lines changed: 44 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
1+
# ETL
2+
13
The OrientDB-ETL module is an amazing tool to move data from and to OrientDB by executing an [ETL process](http://en.wikipedia.org/wiki/Extract,_transform,_load). It's super easy to use. OrientDB ETL is based on the following principles:
2-
- one [configuration file](https://github.com/orientechnologies/orientdb-etl/wiki/Configuration-File) in [JSON](http://en.wikipedia.org/wiki/JSON) format
3-
- one [Extractor](https://github.com/orientechnologies/orientdb-etl/wiki/Extractor) is allowed to extract data from a source
4-
- one [Loader](https://github.com/orientechnologies/orientdb-etl/wiki/Loader) is allowed to load data to a destination
5-
- multiple [Transformers](https://github.com/orientechnologies/orientdb-etl/wiki/Transformer) that transform data in pipeline. They receive something in input, do something, return something as output that will be processed as input by the next component
4+
- one [configuration file](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Configuration-File.html) in [JSON](http://en.wikipedia.org/wiki/JSON) format
5+
- one [Extractor](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Extractor.html) is allowed to extract data from a source
6+
- one [Loader](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Loader.html) is allowed to load data to a destination
7+
- multiple [Transformers](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Transformer.html) that transform data in pipeline. They receive something in input, do something, return something as output that will be processed as input by the next component
68

79
## How ETL works
810
```
@@ -11,11 +13,43 @@ EXTRACTOR => TRANSFORMERS[] => LOADER
1113
Example of a process that extract from a CSV file, apply some change, lookup if the record has already been created and then store the record as document against OrientDB database:
1214

1315
```
14-
+----------------+-----------------------+-----------+
15-
| EXTRACTOR | TRANSFORMERS pipeline | LOADER |
16-
+----------------+-----------------------+-----------+
17-
| FILE ==> CSV->FIELD->MERGE ==> OrientDB |
18-
+----------------+-----------------------+-----------+
16+
+-----------+-----------------------+-----------+
17+
| | PIPELINE |
18+
+ EXTRACTOR +-----------------------+-----------+
19+
| | TRANSFORMERS | LOADER |
20+
+-----------+-----------------------+-----------+
21+
| FILE ==> CSV->FIELD->MERGE ==> OrientDB |
22+
+-----------+-----------------------+-----------+
23+
```
24+
25+
The pipeline, made of transformation and loading phases, can run in parallel by setting the configuration ```{"parallel":true}```.
26+
27+
## Installation
28+
Starting from OrientDB v2.0 the ETL module will be distributed in bundle with the official release. If you want to use it, then follow these steps:
29+
- Clone the repository on your computer, by executing:
30+
- ```git clone https://github.com/orientechnologies/orientdb-etl.git```
31+
- Compile the module, by executing:
32+
- ```mvn clean install```
33+
- Copy ```script/oetl.sh``` (or .bat under Windows) to $ORIENTDB_HOME/bin
34+
- Copy ```target/orientdb-etl-2.0-SNAPSHOT.jar``` to $ORIENTDB_HOME/lib
35+
36+
## Usage
37+
38+
```
39+
$ cd $ORIENTDB_HOME/bin
40+
$ ./oetl.sh config-dbpedia.json
1941
```
2042

21-
Look to the [Documentation](https://github.com/orientechnologies/orientdb-etl/wiki/Home) for more information.
43+
## Available Components
44+
- [Blocks](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Block.html)
45+
- [Sources](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Source.html)
46+
- [Extractors](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Extractor.html)
47+
- [Transformers](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Transformer.html)
48+
- [Loaders](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Loader.html)
49+
50+
Examples:
51+
- [Import DBPedia](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Import-from-DBPedia.html)
52+
- [Import from a DBMS](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Import-from-DBMS.html)
53+
54+
55+
Look to the [Documentation](http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Introduction.html) for more information.

0 commit comments

Comments
 (0)