Skip to content

Commit 2545fc4

Browse files
committed
Add stats to README.md
1 parent 770fc4a commit 2545fc4

File tree

1 file changed

+12
-1
lines changed

1 file changed

+12
-1
lines changed

README.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,17 @@
22
`csvcat` is a very fast csv files compiler (with filtering) written in Go. Using concurrency, `csvcat` can concat and filter
33
a huge number of files without loosing to much in terms of memeroy and processing time.
44

5+
Using a dummy dataset generated from `generate_set.py` with `100` files that have each `100000` lines (around 1.4G) in concurrent and non-concurrent
6+
modes to filter 5 columns out of 10, cvscat take around (on an i7-7820HQ (8)):
7+
```sh
8+
$ ./csvcat --columns "B,A,E,C,F" --delimiter "," --directory "csvset" --concurrency=true
9+
Number of files found: 100
10+
============ Total 3.933102857s ===================
11+
$ ./csvcat --columns "B,A,E,C,F" --delimiter "," --directory "csvset" --concurrency=false
12+
Number of files found: 100
13+
============ Total 10.588019261s ===================
14+
```
15+
516
## Usage of `csvcat`
617
```
718
Usage of ./csvcat:
@@ -24,7 +35,7 @@ Here's an example of how you might run `csvcat` with its flags:
2435
./csvcat --batch 20 --columns "A,B,C" --delimiter "," --directory files
2536
```
2637

27-
`csvcat` expects every csv file to have a header in it's first line where all the columns are labled so that
38+
`csvcat` expects every csv file to have a header in its first line where all the columns are labled so that
2839
it can filter the correct columns. If the csv file is not correctly formated (some lines have more/less columns),
2940
it will try to add an empty column in the correct location.
3041

0 commit comments

Comments
 (0)