Skip to content

Commit ee5f5fa

Browse files
norberttechnorberttech
authored
Added elasticsearch examples (#1384)
1 parent 1c3be54 commit ee5f5fa

File tree

81 files changed

+434
-259
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

81 files changed

+434
-259
lines changed

‎examples/topics/data_frame/cache/output.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
+----------+-----------------------------+---------------------+----------+--------------+-----------+----------------------+
22
| name | html_url | blog | login | public_repos | followers | created_at |
33
+----------+-----------------------------+---------------------+----------+--------------+-----------+----------------------+
4-
| Flow PHP | https://github.com/flow-php | http://flow-php.com | flow-php | 34 | 96 | 2020-10-26T18:40:27Z |
4+
| Flow PHP | https://github.com/flow-php | http://flow-php.com | flow-php | 38 | 109 | 2020-10-26T18:40:27Z |
55
+----------+-----------------------------+---------------------+----------+--------------+-----------+----------------------+
66
1 rows

‎examples/topics/data_frame/cache/output/.gitignore‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1 @@
1-
*
2-
!.gitignore
1+
*

‎examples/topics/data_frame/reorder_columns/output.txt‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
+--------------------------------------+-------+-------+-------+--------+--------+---------+---------+---------------------------+---------------------------+----------+----------+---------+-------------------------+----------------------+----------------------------------------------------------------------------------------------------+
2-
| uuid | int_a | int_b | bool | bool_a | bool_c | float_a | float_b | datetime_d | datetime_z | string_a | string_b | list | json | map | struct |
3-
+--------------------------------------+-------+-------+-------+--------+--------+---------+---------+---------------------------+---------------------------+----------+----------+---------+-------------------------+----------------------+----------------------------------------------------------------------------------------------------+
4-
| 06143adb-3009-45c8-a4f0-c7016f97cab7 | 1 | 1 | false | false | false | 572.91 | 210.21 | 2024-04-01T00:00:00+00:00 | 2024-04-01T00:00:00+00:00 | string | string | [1,2,3] | {"id":1,"status":"NEW"} | ["zero","one","two"] | {"street":"street","city":"city","zip":"zip","country":"country","location":{"lat":1.5,"lon":1.5}} |
5-
+--------------------------------------+-------+-------+-------+--------+--------+---------+---------+---------------------------+---------------------------+----------+----------+---------+-------------------------+----------------------+----------------------------------------------------------------------------------------------------+
1+
+--------------------------------------+-------+-------+-------+--------+--------+------------+------------+---------------------------+---------------------------+----------+----------+---------+-------------------------+----------------------+----------------------------------------------------------------------------------------------------+
2+
| uuid | int_a | int_b | bool | bool_a | bool_c | float_a | float_b | datetime_d | datetime_z | string_a | string_b | list | json | map | struct |
3+
+--------------------------------------+-------+-------+-------+--------+--------+------------+------------+---------------------------+---------------------------+----------+----------+---------+-------------------------+----------------------+----------------------------------------------------------------------------------------------------+
4+
| 06143adb-3009-45c8-a4f0-c7016f97cab7 | 1 | 1 | false | false | false | 572.910000 | 210.210000 | 2024-04-01T00:00:00+00:00 | 2024-04-01T00:00:00+00:00 | string | string | [1,2,3] | {"id":1,"status":"NEW"} | ["zero","one","two"] | {"street":"street","city":"city","zip":"zip","country":"country","location":{"lat":1.5,"lon":1.5}} |
5+
+--------------------------------------+-------+-------+-------+--------+--------+------------+------------+---------------------------+---------------------------+----------+----------+---------+-------------------------+----------------------+----------------------------------------------------------------------------------------------------+
66
1 rows

‎examples/topics/data_reading/database/output.txt‎

Lines changed: 24 additions & 24 deletions
Large diffs are not rendered by default.

‎examples/topics/data_reading/elasticsearch/.env.dist‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
ELASTICSEARCH_URL=localhost:9200

‎examples/topics/data_reading/elasticsearch/code.php‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
use function Flow\ETL\Adapter\Elasticsearch\{entry_id_factory, es_hits_to_rows, from_es, to_es_bulk_index};
6+
use function Flow\ETL\DSL\{data_frame, from_array, to_stream};
7+
use Symfony\Component\Dotenv\Dotenv;
8+
9+
require __DIR__ . '/../../../autoload.php';
10+
11+
if (!\file_exists(__DIR__ . '/.env')) {
12+
print 'Example skipped. Please create .env file with Azure Storage Account credentials.' . PHP_EOL;
13+
14+
return;
15+
}
16+
17+
$dotenv = new Dotenv();
18+
$dotenv->load(__DIR__ . '/.env');
19+
20+
data_frame()
21+
->read(from_array([
22+
['id' => 1, 'text' => 'lorem ipsum'],
23+
['id' => 2, 'text' => 'lorem ipsum'],
24+
['id' => 3, 'text' => 'lorem ipsum'],
25+
['id' => 4, 'text' => 'lorem ipsum'],
26+
['id' => 5, 'text' => 'lorem ipsum'],
27+
['id' => 6, 'text' => 'lorem ipsum'],
28+
]))
29+
->write(
30+
to_es_bulk_index(
31+
[
32+
'hosts' => [$_ENV['ELASTICSEARCH_URL']],
33+
],
34+
$index = 'test_index',
35+
entry_id_factory('id')
36+
)
37+
)
38+
->run();
39+
40+
data_frame()
41+
->read(from_es(
42+
[
43+
'hosts' => [$_ENV['ELASTICSEARCH_URL']],
44+
],
45+
[
46+
'index' => $index,
47+
'body' => [
48+
'query' => [
49+
'match_all' => ['boost' => 1.0],
50+
],
51+
],
52+
]
53+
))
54+
->write(to_stream(__DIR__ . '/output.raw.txt', truncate: false))
55+
->transform(es_hits_to_rows())
56+
->write(to_stream(__DIR__ . '/output.txt', truncate: false))
57+
->run();

‎examples/topics/data_reading/elasticsearch/output.raw.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
+------+-----------+---------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2+
| took | timed_out | _shards | hits |
3+
+------+-----------+---------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
4+
| 0 | false | {"total":1,"successful":1,"skipped":0,"failed":0} | {"total":{"value":6,"relation":"eq"},"max_score":1,"hits":[{"_index":"test_index","_type":"_doc","_id":"1","_score":1,"_source":{"id":1,"text":"lorem ipsum"}},{"_index":"test_index","_type":"_doc","_id":"2","_score":1,"_source":{"id":2,"text":"lorem ipsum"}},{"_index":"test_index","_type":"_doc","_id":"3","_score":1,"_source":{"id":3,"text":"lorem ipsum"}},{"_index":"test_index","_type":"_doc","_id":"4","_score":1,"_source":{"id":4,"text":"lorem ipsum"}},{"_index":"test_index","_type":"_doc","_id":"5","_score":1,"_source":{"id":5,"text":"lorem ipsum"}},{"_index":"test_index","_type":"_doc","_id":"6","_score":1,"_source":{"id":6,"text":"lorem ipsum"}}]} |
5+
+------+-----------+---------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
6+
1 rows

‎examples/topics/data_reading/elasticsearch/output.txt‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
+----+-------------+
2+
| id | text |
3+
+----+-------------+
4+
| 1 | lorem ipsum |
5+
| 2 | lorem ipsum |
6+
| 3 | lorem ipsum |
7+
| 4 | lorem ipsum |
8+
| 5 | lorem ipsum |
9+
| 6 | lorem ipsum |
10+
+----+-------------+
11+
6 rows

‎examples/topics/data_reading/http_dynamic/output.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
+----------+-----------------------------+---------------------+----------+--------------+-----------+----------------------+
22
| name | html_url | blog | login | public_repos | followers | created_at |
33
+----------+-----------------------------+---------------------+----------+--------------+-----------+----------------------+
4-
| Flow PHP | https://github.com/flow-php | http://flow-php.com | flow-php | 37 | 106 | 2020-10-26T18:40:27Z |
4+
| Flow PHP | https://github.com/flow-php | http://flow-php.com | flow-php | 38 | 109 | 2020-10-26T18:40:27Z |
55
+----------+-----------------------------+---------------------+----------+--------------+-----------+----------------------+
66
1 rows

‎examples/topics/data_writing/elasticsearch/.env.dist‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
ELASTICSEARCH_URL=localhost:9200

0 commit comments

Comments
 (0)