Skip to content

Commit 00d75d3

Browse files
vga91vga91
authored
Fixes #4253: Missing apache arrow/jsonParams documentation (#4255)
* Fixes #4253: Missing apache arrow/jsonParams documentation * Added missing apache arrow/jsonParams documentation * fix typos * changes review
1 parent 16ff77f commit 00d75d3

24 files changed

+876
-5
lines changed

‎docs/asciidoc/modules/ROOT/nav.adoc‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,13 @@ include::partial$generated-documentation/nav.adoc[]
2828
** xref::import/html.adoc[]
2929
** xref::import/parquet.adoc[]
3030
** xref::import/gexf.adoc[]
31+
** xref::import/arrow.adoc[]
32+
** xref::import/load-json.adoc[]
3133
3234
* xref:export/index.adoc[]
3335
** xref::export/xls.adoc[]
36+
** xref::export/arrow.adoc[]
37+
3438
3539
* xref:database-integration/index.adoc[]
3640
** xref::database-integration/load-jdbc.adoc[]

‎docs/asciidoc/modules/ROOT/pages/export/arrow.adoc‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
[[export-arrow]]
2+
= Export to Apache Arrow
3+
:description: This section describes procedures that can be used to export data in Apache Arrow format.
4+
5+
These procedures export data into a format that's used by many Apache and non-Apache tools.
6+
7+
8+
[[export-arrow-available-procedures]]
9+
== Available Procedures
10+
11+
The table below describes the available procedures:
12+
13+
[separator=¦,opts=header,cols="5,1m"]
14+
|===
15+
¦Qualified Name¦Type
16+
¦xref::overview/apoc.export/apoc.export.arrow.all.adoc[apoc.export.arrow.all icon:book[]] +
17+
`apoc.export.arrow.all(file STRING, config MAP<STRING, ANY>)` - exports the full database as an arrow file.
18+
¦label:procedure[]
19+
20+
¦xref::overview/apoc.export/apoc.export.arrow.graph.adoc[apoc.export.arrow.graph.adoc icon:book[]] +
21+
`apoc.export.arrow.graph(file STRING, graph ANY, config MAP<STRING, ANY>)` - exports the given graph as an arrow file.
22+
¦label:procedure[]
23+
24+
¦xref::overview/apoc.export/apoc.export.arrow.query.adoc[apoc.export.arrow.query.adoc icon:book[]] +
25+
`apoc.export.arrow.stream.all(config MAP<STRING, ANY>)` - exports the full database as an arrow byte array.
26+
¦label:procedure[]
27+
28+
¦xref::overview/apoc.export/apoc.export.arrow.stream.all.adoc[apoc.export.arrow.stream.all icon:book[]] +
29+
`apoc.export.arrow.all(file STRING, config MAP<STRING, ANY>)` - exports the full database as an arrow file.
30+
¦label:procedure[]
31+
32+
¦xref::overview/apoc.export/apoc.export.arrow.stream.graph.adoc[apoc.export.arrow.stream.graph.adoc icon:book[]] +
33+
`apoc.export.arrow.stream.graph(graph ANY, config MAP<STRING, ANY>)` - exports the given graph as an arrow byte array.
34+
¦label:procedure[]
35+
36+
¦xref::overview/apoc.export/apoc.export.arrow.stream.query.adoc[apoc.export.arrow.stream.query.adoc icon:book[]] +
37+
`apoc.export.arrow.stream.query(query ANY, config MAP<STRING, ANY>)` - exports the given Cypher query as an arrow byte array.
38+
¦label:procedure[]
39+
|===
40+
41+
42+
[[export-arrow-file-export]]
43+
== Exporting to a file
44+
45+
include::partial$enableFileExport.adoc[]
46+
47+
48+
[[export-arrow-examples]]
49+
== Examples
50+
51+
52+
[[export-cypher-query-arrow]]
53+
=== Export results of Cypher query to Apache Arrow file
54+
55+
[source,cypher]
56+
----
57+
CALL apoc.export.arrow.query('query_test.arrow',
58+
"RETURN 1 AS intData, 'a' AS stringData,
59+
true AS boolData,
60+
[1, 2, 3] AS intArray,
61+
[1.1, 2.2, 3.3] AS doubleArray,
62+
[true, false, true] AS boolArray,
63+
[1, '2', true, null] AS mixedArray,
64+
{foo: 'bar'} AS mapData,
65+
localdatetime('2015-05-18T19:32:24') as dateData,
66+
[[0]] AS arrayArray,
67+
1.1 AS doubleData"
68+
) YIELD file
69+
----
70+
71+
.Results
72+
[opts="header"]
73+
|===
74+
| file | source | format | nodes | relationships | properties | time | rows | batchSize | batches | done | data
75+
| "query_test.arrow" | "statement: cols(11)" | "arrow" | 0 | 0 | 11 | 468 | 11 | 2000 | 1 | true | <null>
76+
|===
77+
78+
79+
80+
[[export-cypher-query-arrow-stream]]
81+
=== Export results of Cypher query to Apache Arrow binary output
82+
83+
[source,cypher]
84+
----
85+
CALL apoc.export.arrow.stream.query('query_test.arrow',
86+
"RETURN 1 AS intData, 'a' AS stringData,
87+
true AS boolData,
88+
[1, 2, 3] AS intArray,
89+
[1.1, 2.2, 3.3] AS doubleArray,
90+
[true, false, true] AS boolArray,
91+
[1, '2', true, null] AS mixedArray,
92+
{foo: 'bar'} AS mapData,
93+
localdatetime('2015-05-18T19:32:24') as dateData,
94+
[[0]] AS arrayArray,
95+
1.1 AS doubleData"
96+
) YIELD value
97+
----
98+
99+
.Results
100+
[opts="header"]
101+
|===
102+
| value
103+
| <binary Apache Arrow output>
104+
|===

‎docs/asciidoc/modules/ROOT/pages/export/index.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,5 @@ For more information on how to use these procedures, see:
1515

1616
* xref::export/xls.adoc[]
1717
* xref::export/parquet.adoc[]
18+
* xref::export/arrow.adoc[]
19+

‎docs/asciidoc/modules/ROOT/pages/import/arrow.adoc‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
[[load-arrow]]
2+
= Load and Import Arrow
3+
:description: This section describes procedures that can be used to import Apache Arrow data from web APIs or files.
4+
5+
6+
7+
The following procedures allow you to read an Apache Arrow file exported via xref::export/arrow.adoc[apoc.export.arrow.* procedures].
8+
It could also potentially read other Apache Arrow files that have not been created via the export procedures.
9+
10+
[[load-arrow-available-procedures]]
11+
== Procedure and Function Overview
12+
13+
The table below describes the available procedures and functions:
14+
15+
[separator=¦,opts=header,cols="5,1m"]
16+
|===
17+
¦Qualified Name¦Type
18+
¦xref::overview/apoc.load/apoc.load.arrow.adoc[apoc.load.arrow icon:book[]] +
19+
`apoc.load.arrow(file STRING, config MAP<STRING, ANY>)` - loads values from the provided Arrow file.
20+
¦label:procedure[]
21+
22+
¦xref::overview/apoc.load/apoc.load.arrow.stream.adoc[apoc.load.arrow.stream.adoc icon:book[]] +
23+
`apoc.load.arrow.stream(source LIST<INTEGER>, config MAP<STRING, ANY>)` - loads values from the provided Arrow byte array.
24+
¦label:procedure[]
25+
26+
¦xref::overview/apoc.import/apoc.import.arrow.adoc[apoc.import.arrow icon:book[]] +
27+
`apoc.import.arrow(urlOrBinaryFile ANY, config MAP<STRING, ANY>)` - imports entities from the provided Arrow file or byte array
28+
¦label:procedure[]
29+
|===
30+
31+
32+
33+
[[load-arrow-available-procedures-apoc.load.arrow]]
34+
=== `apoc.load.arrow`
35+
36+
37+
This procedure takes a file or HTTP URL and parses the Apache Arrow into a map data structure.
38+
39+
[separator=¦,opts=header,cols="1m"]
40+
|===
41+
¦signature
42+
¦apoc.load.arrow(file :: STRING, config = {} :: MAP) :: (value :: MAP)
43+
|===
44+
45+
46+
Currently, this procedure does not support any config parameters.
47+
48+
include::includes/enableFileImport.adoc[]
49+
50+
[[load-arrow-available-procedures-apoc.load.arrow.stream]]
51+
=== `apoc.load.arrow.stream`
52+
53+
54+
This procedure takes a byte[] source and parses the Apache Arrow into a map data structure.
55+
56+
[separator=¦,opts=header,cols="1m"]
57+
|===
58+
¦signature
59+
¦apoc.load.arrow.stream(source :: LIST<INTEGER>, config = {} :: MAP) :: (value :: MAP)
60+
|===
61+
62+
Currently, this procedure does not support any config parameters.
63+
64+
65+
[[load-arrow-examples]]
66+
== Examples
67+
68+
The following section contains examples showing how to import data from various Apache Arrow sources.
69+
70+
[[load-arrow-examples-local-file]]
71+
=== Import from local file
72+
73+
Taking the output xref::export/arrow.adoc#export-cypher-query-arrow[of this case]:
74+
75+
.The following query processes a `test.arrow` file and returns the content as Cypher data structures
76+
[source,cypher]
77+
----
78+
CALL apoc.load.arrow('test.arrow')
79+
YIELD value
80+
RETURN value
81+
----
82+
83+
.Results
84+
[options="header"]
85+
|===
86+
| value
87+
| {arrayArray -> ["[0]"], dateData -> 2015-05-18T19:32:24Z, boolArray -> [true,false,true], intArray -> [1,2,3], mapData -> "{"foo":"bar"}", boolData -> true, intData -> 1, mixedArray -> ["1","2","true",<null>], doubleArray -> [1.1,2.2,3.3], doubleData -> 1.1, stringData -> "a"}
88+
|===
89+
90+
91+
[[load-arrow-examples-binary-source]]
92+
=== Import from binary source
93+
94+
Taking the output xref::export/arrow.adoc#export-cypher-query-arrow-stream[of this case]:
95+
96+
.The following query processes a `test.arrow` file and returns the content as Cypher data structures
97+
[source,cypher]
98+
----
99+
CALL apoc.load.arrow.stream('<binary arrow file>')
100+
YIELD value
101+
RETURN value
102+
----
103+
104+
.Results
105+
[options="header"]
106+
|===
107+
| value
108+
| {arrayArray -> ["[0]"], dateData -> 2015-05-18T19:32:24Z, boolArray -> [true,false,true], intArray -> [1,2,3], mapData -> "{"foo":"bar"}", boolData -> true, intData -> 1, mixedArray -> ["1","2","true",<null>], doubleArray -> [1.1,2.2,3.3], doubleData -> 1.1, stringData -> "a"}
109+
|===
110+
111+
112+
[[import-arrow]]
113+
=== Import Arrow file created by Export Arrow procedures
114+
115+
The `apoc.import.arrow` procedure can be used to import Apache Arrow files created by the xref::export/arrow.adoc[apoc.export.arrow.*] procedures.
116+
117+
This procedure should not be confused with the `apoc.load.arrow*` procedures,
118+
which just loads the values of the Arrow file, and does not create entities in the database.
119+
120+
See xref::overview/apoc.import/apoc.import.arrow.adoc[this page] for more info.
121+

‎docs/asciidoc/modules/ROOT/pages/import/index.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,5 @@ For more information on these procedures, see:
1414
* xref::import/html.adoc[]
1515
* xref::import/parquet.adoc[]
1616
* xref::import/gexf.adoc[]
17+
* xref::import/load-json.adoc[]
18+
* xref::import/arrow.adoc[]

‎docs/asciidoc/modules/ROOT/pages/import/load-json.adoc‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
[[apoc.load.jsonParams]]
2+
= Load JSON
3+
:description: This section contains reference documentation for the apoc.load.jsonParams procedure.
4+
5+
label:procedure[] label:apoc-core[]
6+
7+
[.emphasis]
8+
====
9+
apoc.load.jsonParams('urlOrBinary',\{headers\}, \{payload\}, $config) YIELD value
10+
11+
It loads from JSON URL, a RestAPI endpoint URL, or from a binary source.
12+
13+
The headers and the payload parameters can be used to execute a RestAPI.
14+
The procedure will return multiple values if the starting JSON is a list of JSON or a single result in case it is a single JSON.
15+
====
16+
17+
== Signature
18+
19+
[source]
20+
----
21+
apoc.load.jsonParams(urlOrKeyOrBinary :: ANY?, headers :: MAP?, payload :: STRING?, path = :: STRING?, config = {} :: MAP?) :: (value :: MAP?)
22+
----
23+
24+
== Input parameters
25+
[.procedures, opts=header]
26+
|===
27+
| Name | Type | Default
28+
|urlOrKeyOrBinary|ANY?|null
29+
|headers|MAP?|null
30+
|payload|STRING?|null
31+
|path|STRING?|
32+
|config|MAP?|{}
33+
|===
34+
35+
== Output parameters
36+
[.procedures, opts=header]
37+
|===
38+
| Name | Type
39+
|value|MAP?
40+
|===
41+
42+
== Reading from a file
43+
include::includes/enableFileImport.adoc[]
44+
45+
[[usage-apoc.load.jsonParams]]
46+
== Usage Examples
47+
include::partial$usage/apoc.load.jsonParams.adoc[]
48+

‎docs/asciidoc/modules/ROOT/pages/overview/apoc.export/apoc.export.arrow.all.adoc‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
= apoc.export.arrow.all
2+
:description: This section contains reference documentation for the apoc.export.arrow.all procedure.
3+
4+
label:procedure[]
5+
6+
[.emphasis]
7+
`apoc.export.arrow.all(file STRING, config MAP<STRING, ANY>)` - exports the full database as an arrow file.
8+
9+
[NOTE]
10+
====
11+
It is considered unsafe to run this procedure using multiple threads.
12+
It is therefore not supported by the parallel runtime (introduced in Neo4j 5.13).
13+
For more information, see the link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/planning-and-tuning/runtimes/concepts#runtimes-parallel-runtime[Cypher Manual -> Parallel runtime].
14+
====
15+
16+
== Signature
17+
18+
[source]
19+
----
20+
apoc.export.arrow.all(file :: STRING, config = {} :: MAP) :: (file :: STRING, source :: STRING, format :: STRING, nodes :: INTEGER, relationships :: INTEGER, properties :: INTEGER, time :: INTEGER, rows :: INTEGER, batchSize :: INTEGER, batches :: INTEGER, done :: BOOLEAN, data :: ANY)
21+
----
22+
23+
== Input parameters
24+
[.procedures, opts=header]
25+
|===
26+
| Name | Type | Default
27+
|file|STRING|null
28+
|config|MAP|null
29+
|===
30+
31+
== Output parameters
32+
[.procedures, opts=header]
33+
|===
34+
| Name | Type
35+
|file|STRING
36+
|source|STRING
37+
|format|STRING
38+
|nodes|INTEGER
39+
|relationships|INTEGER
40+
|properties|INTEGER
41+
|time|INTEGER
42+
|rows|INTEGER
43+
|batchSize|INTEGER
44+
|batches|INTEGER
45+
|done|BOOLEAN
46+
|data|STRING
47+
|===
48+
49+
== Config parameters
50+
include::partial$usage/config/apoc.export.arrow.all.adoc[]
51+
52+
[[usage-apoc.export.arrow.all]]
53+
== Usage Examples
54+
include::partial$usage/apoc.export.arrow.all.adoc[]
55+
56+
xref::export/arrow.adoc[More documentation of apoc.export.arrow.all,role=more information]

0 commit comments

Comments
 (0)