Skip to content

Commit 7f45bc2

Browse files
committed
Initial import
0 parents  commit 7f45bc2

File tree

12 files changed

+1135
-0
lines changed

12 files changed

+1135
-0
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
vendor/
2+
/composer.lock

.travis.yml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
language: php
2+
3+
php:
4+
# - 5.3 # requires old distro
5+
- 5.4
6+
- 5.5
7+
- 5.6
8+
- 7.0
9+
- 7.1
10+
- 7.2
11+
- hhvm # ignore errors, see below
12+
13+
# lock distro so new future defaults will not break the build
14+
dist: trusty
15+
16+
matrix:
17+
include:
18+
- php: 5.3
19+
dist: precise
20+
allow_failures:
21+
- php: hhvm
22+
23+
sudo: false
24+
25+
install:
26+
- composer install --no-interaction
27+
28+
script:
29+
- vendor/bin/phpunit --coverage-text

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
The MIT License (MIT)
2+
3+
Copyright (c) 2018 Christian Lück
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is furnished
10+
to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21+
THE SOFTWARE.

README.md

Lines changed: 270 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,270 @@
1+
# clue/reactphp-csv [![Build Status](https://travis-ci.org/clue/reactphp-csv.svg?branch=master)](https://travis-ci.org/clue/reactphp-csv)
2+
3+
Streaming CSV (Comma-Separated Values or Character-Separated Values) parser and encoder for [ReactPHP](https://reactphp.org/).
4+
5+
CSV (Comma-Separated Values or less commonly Character-Separated Values) can be
6+
used to store a large number of (uniform) records in simple text-based files,
7+
such as a list of user records or log entries. CSV is not exactly a new format
8+
and has been used in a large number of systems for decades. In particular, CSV
9+
is often used for historical reasons and despite its shortcomings, it is still a
10+
very common export format for a large number of tools to interface with
11+
spreadsheet processors (such as Exel, Calc etc.). This library provides a simple
12+
streaming API to process very large CSV files with thousands or even millions of
13+
rows efficiently without having to load the whole file into memory at once.
14+
15+
* **Standard interfaces** -
16+
Allows easy integration with existing higher-level components by implementing
17+
ReactPHP's standard streaming interfaces.
18+
* **Lightweight, SOLID design** -
19+
Provides a thin abstraction that is [*just good enough*](https://en.wikipedia.org/wiki/Principle_of_good_enough)
20+
and does not get in your way.
21+
Builds on top of well-tested components and well-established concepts instead of reinventing the wheel.
22+
* **Good test coverage** -
23+
Comes with an [automated tests suite](#tests) and is regularly tested in the *real world*
24+
25+
**Table of contents**
26+
27+
* [Usage](#usage)
28+
* [Decoder](#decoder)
29+
* [Encoder](#encoder)
30+
* [Install](#install)
31+
* [Tests](#tests)
32+
* [License](#license)
33+
* [More](#more)
34+
35+
## Usage
36+
37+
### Decoder
38+
39+
The `Decoder` (parser) class can be used to make sure you only get back
40+
complete, valid CSV elements when reading from a stream.
41+
It wraps a given
42+
[`ReadableStreamInterface`](https://github.com/reactphp/stream#readablestreaminterface)
43+
and exposes its data through the same interface, but emits the CSV elements
44+
as parsed values instead of just chunks of strings:
45+
46+
```
47+
test,1,24
48+
"hello world",2,48
49+
```
50+
```php
51+
$stdin = new ReadableResourceStream(STDIN, $loop);
52+
53+
$stream = new Decoder($stdin);
54+
55+
$stream->on('data', function ($data) {
56+
// data is a parsed element from the CSV stream
57+
// line 1: $data = array('test', '1', '24');
58+
// line 2: $data = array('hello world', '2', '48');
59+
var_dump($data);
60+
});
61+
```
62+
63+
ReactPHP's streams emit chunks of data strings and make no assumption about their lengths.
64+
These chunks do not necessarily represent complete CSV elements, as an
65+
element may be broken up into multiple chunks.
66+
This class reassembles these elements by buffering incomplete ones.
67+
68+
The `Decoder` supports the same optional parameters as the underlying
69+
[`str_getcsv()`](http://php.net/str_getcsv) function.
70+
This means that, by default, CSV fields will be delimited by a comma (`,`), will
71+
use a quote enclosure character (`"`) and a backslash escape character (`\`).
72+
This behavior can be controlled through the optional constructor parameters:
73+
74+
```php
75+
$stream = new Decoder($stdin, ';');
76+
77+
$stream->on('data', function ($data) {
78+
// CSV fields will now be delimited by semicolon
79+
});
80+
```
81+
82+
Additionally, the `Decoder` limits the maximum buffer size (maximum line
83+
length) to avoid buffer overflows due to malformed user input. Usually, there
84+
should be no need to change this value, unless you know you're dealing with some
85+
unreasonably long lines. It accepts an additional argument if you want to change
86+
this from the default of 64 KiB:
87+
88+
```php
89+
$stream = new Decoder($stdin, ',', '"', '\\', 64 * 1024);
90+
```
91+
92+
If the underlying stream emits an `error` event or the plain stream contains
93+
any data that does not represent a valid CSV stream,
94+
it will emit an `error` event and then `close` the input stream:
95+
96+
```php
97+
$stream->on('error', function (Exception $error) {
98+
// an error occured, stream will close next
99+
});
100+
```
101+
102+
If the underlying stream emits an `end` event, it will flush any incomplete
103+
data from the buffer, thus either possibly emitting a final `data` event
104+
followed by an `end` event on success or an `error` event for
105+
incomplete/invalid CSV data as above:
106+
107+
```php
108+
$stream->on('end', function () {
109+
// stream successfully ended, stream will close next
110+
});
111+
```
112+
113+
If either the underlying stream or the `Decoder` is closed, it will forward
114+
the `close` event:
115+
116+
```php
117+
$stream->on('close', function () {
118+
// stream closed
119+
// possibly after an "end" event or due to an "error" event
120+
});
121+
```
122+
123+
The `close(): void` method can be used to explicitly close the `Decoder` and
124+
its underlying stream:
125+
126+
```php
127+
$stream->close();
128+
```
129+
130+
The `pipe(WritableStreamInterface $dest, array $options = array(): WritableStreamInterface`
131+
method can be used to forward all data to the given destination stream.
132+
Please note that the `Decoder` emits decoded/parsed data events, while many
133+
(most?) writable streams expect only data chunks:
134+
135+
```php
136+
$stream->pipe($logger);
137+
```
138+
139+
For more details, see ReactPHP's
140+
[`ReadableStreamInterface`](https://github.com/reactphp/stream#readablestreaminterface).
141+
142+
### Encoder
143+
144+
The `Encoder` (serializer) class can be used to make sure anything you write to
145+
a stream ends up as valid CSV elements in the resulting CSV stream.
146+
It wraps a given
147+
[`WritableStreamInterface`](https://github.com/reactphp/stream#writablestreaminterface)
148+
and accepts its data through the same interface, but handles any data as complete
149+
CSV elements instead of just chunks of strings:
150+
151+
```php
152+
$stdout = new WritableResourceStream(STDOUT, $loop);
153+
154+
$stream = new Encoder($stdout);
155+
156+
$stream->write(array('test', true, 24));
157+
$stream->write(array('hello world', 2, 48));
158+
```
159+
```
160+
test,1,24
161+
"hello world",2,48
162+
```
163+
164+
The `Encoder` supports the same optional parameters as the underlying
165+
[`fputcsv()`](http://php.net/fputcsv) function.
166+
This means that, by default, CSV fields will be delimited by a comma (`,`), will
167+
use a quote enclosure character (`"`) and a backslash escape character (`\`).
168+
This behavior can be controlled through the optional constructor parameters:
169+
170+
```php
171+
$stream = new Encoder($stdout, ';');
172+
173+
$stream->write(array('hello', 'world'));
174+
```
175+
```
176+
hello;world
177+
```
178+
179+
If the underlying stream emits an `error` event or the given data contains
180+
any data that can not be represented as a valid CSV stream,
181+
it will emit an `error` event and then `close` the input stream:
182+
183+
```php
184+
$stream->on('error', function (Exception $error) {
185+
// an error occured, stream will close next
186+
});
187+
```
188+
189+
If either the underlying stream or the `Encoder` is closed, it will forward
190+
the `close` event:
191+
192+
```php
193+
$stream->on('close', function () {
194+
// stream closed
195+
// possibly after an "end" event or due to an "error" event
196+
});
197+
```
198+
199+
The `end(mixed $data = null): void` method can be used to optionally emit
200+
any final data and then soft-close the `Encoder` and its underlying stream:
201+
202+
```php
203+
$stream->end();
204+
```
205+
206+
The `close(): void` method can be used to explicitly close the `Encoder` and
207+
its underlying stream:
208+
209+
```php
210+
$stream->close();
211+
```
212+
213+
For more details, see ReactPHP's
214+
[`WritableStreamInterface`](https://github.com/reactphp/stream#writablestreaminterface).
215+
216+
## Install
217+
218+
The recommended way to install this library is [through Composer](https://getcomposer.org).
219+
[New to Composer?](https://getcomposer.org/doc/00-intro.md)
220+
221+
This will install the latest supported version:
222+
223+
```bash
224+
$ composer require clue/reactphp-csv:dev-master
225+
```
226+
227+
This project aims to run on any platform and thus does not require any PHP
228+
extensions and supports running on legacy PHP 5.3 through current PHP 7+ and
229+
HHVM.
230+
It's *highly recommended to use PHP 7+* for this project.
231+
232+
## Tests
233+
234+
To run the test suite, you first need to clone this repo and then install all
235+
dependencies [through Composer](https://getcomposer.org):
236+
237+
```bash
238+
$ composer install
239+
```
240+
241+
To run the test suite, go to the project root and run:
242+
243+
```bash
244+
$ php vendor/bin/phpunit
245+
```
246+
247+
## License
248+
249+
This project is released under the permissive [MIT license](LICENSE).
250+
251+
> Did you know that I offer custom development services and issuing invoices for
252+
sponsorships of releases and for contributions? Contact me (@clue) for details.
253+
254+
## More
255+
256+
* If you want to learn more about processing streams of data, refer to the documentation of
257+
the underlying [react/stream](https://github.com/reactphp/stream) component.
258+
259+
* If you want to process compressed CSV files (`.csv.gz` file extension)
260+
you may want to use [clue/reactphp-zlib](https://github.com/clue/reactphp-zlib)
261+
on the compressed input stream before passing the decompressed stream to the CSV decoder.
262+
263+
* If you want to create compressed CSV files (`.csv.gz` file extension)
264+
you may want to use [clue/reactphp-zlib](https://github.com/clue/reactphp-zlib)
265+
on the resulting CSV encoder output stream before passing the compressed
266+
stream to the file output stream.
267+
268+
* If you want to concurrently process the records from your CSV stream,
269+
you may want to use [clue/reactphp-flux](https://github.com/clue/reactphp-flux)
270+
to concurrently process many (but not too many) records at once.

composer.json

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"name": "clue/reactphp-csv",
3+
"description": "Streaming CSV (Comma-Separated Values or Character-Separated Values) parser and encoder for ReactPHP",
4+
"keywords": ["CSV", "comma-separated values", "character-separated values", "streaming", "ReactPHP"],
5+
"homepage": "https://github.com/clue/reactphp-csv",
6+
"license": "MIT",
7+
"authors": [
8+
{
9+
"name": "Christian Lück",
10+
"email": "[email protected]"
11+
}
12+
],
13+
"autoload": {
14+
"psr-4": { "Clue\\React\\Csv\\": "src/" }
15+
},
16+
"require": {
17+
"php": ">=5.3",
18+
"react/stream": "^1.0 || ^0.7 || ^0.6"
19+
},
20+
"require-dev": {
21+
"react/event-loop": "^1.0 || ^0.5 || ^0.4 || ^0.3",
22+
"phpunit/phpunit": "^6.0 || ^5.7 || ^4.8.35"
23+
}
24+
}

examples/validate.php

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
<?php
2+
3+
use React\EventLoop\Factory;
4+
use React\Stream\ReadableResourceStream;
5+
use React\Stream\WritableResourceStream;
6+
use Clue\React\Csv\Decoder;
7+
use Clue\React\Csv\Encoder;
8+
9+
require __DIR__ . '/../vendor/autoload.php';
10+
11+
$loop = Factory::create();
12+
13+
$exit = 0;
14+
$in = new ReadableResourceStream(STDIN, $loop);
15+
$out = new WritableResourceStream(STDOUT, $loop);
16+
$info = new WritableResourceStream(STDERR, $loop);
17+
18+
$decoder = new Decoder($in);
19+
$encoder = new Encoder($out);
20+
$decoder->pipe($encoder);
21+
22+
$decoder->on('error', function (Exception $e) use ($info, &$exit) {
23+
$info->write('ERROR: ' . $e->getMessage() . PHP_EOL);
24+
$exit = 1;
25+
});
26+
27+
$info->write('You can pipe/write a valid CSV stream to STDIN' . PHP_EOL);
28+
$info->write('Valid CSV will be forwarded to STDOUT' . PHP_EOL);
29+
$info->write('Invalid CSV will raise an error on STDERR and exit with code 1' . PHP_EOL);
30+
31+
$loop->run();
32+
33+
exit($exit);

0 commit comments

Comments
 (0)