You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Property graph is a widely-adopted model for knowledge graphs, where both nodes and relationships can have properties.
32
+
28
33
[Graph database concepts](https://neo4j.com/docs/getting-started/appendix/graphdb-concepts/) has a good introduction to basic concepts of property graphs.
29
34
35
+
The following concepts will be used in the following sections:
*[Node label](https://neo4j.com/docs/getting-started/appendix/graphdb-concepts/#graphdb-labels), which represents a type of nodes.
38
+
*[Relationship](https://neo4j.com/docs/getting-started/appendix/graphdb-concepts/#graphdb-relationship), which describes a connection between two nodes.
*[Properties](https://neo4j.com/docs/getting-started/appendix/graphdb-concepts/#graphdb-properties), which are key-value pairs associated with nodes and relationships.
41
+
42
+
### Data Mapping
43
+
44
+
Data from collectors are mapped to graph elements in various types:
45
+
46
+
1. Rows from collectors → Nodes in the graph
47
+
2. Rows from collectors → Relationships in the graph (including source and target nodes of the relationship)
48
+
49
+
This is what you need to provide to define these mappings:
50
+
51
+
* Specify [nodes to export](#nodes-to-export).
52
+
*[Declare extra node labels](#declare-extra-node-labels), for labels to appear as source/target nodes of relationships but not exported as nodes.
53
+
* Specify [relationships to export](#relationships-to-export).
54
+
55
+
In addition, the same node may appear multiple times, from exported nodes and various relationships.
56
+
They should appear as the same node in the target graph database.
57
+
CocoIndex automatically [matches and deduplicates nodes](#nodes-matching-and-deduplicating) based on their primary key values.
58
+
59
+
### Nodes to Export
60
+
61
+
Here's how CocoIndex data elements map to nodes in the graph:
62
+
63
+
| CocoIndex Element | Graph Element |
64
+
|-------------------|------------------|
65
+
| an export target | nodes with a unique label |
66
+
| a collected row | a node |
67
+
| a field | a property of node |
68
+
69
+
Note that the label used in different `Nodes`s should be unique.
70
+
71
+
`cocoindex.targets.Nodes` is to describe mapping to nodes. It has the following fields:
72
+
73
+
*`label` (`str`): The label of the node.
74
+
75
+
For example, consider we have collected the following rows:
76
+
77
+
<small>
78
+
79
+
| filename | summary |
80
+
|----------|---------|
81
+
| chapter1.md | At the beginning, ... |
82
+
| chapter2.md | In the second day, ... |
83
+
84
+
</small>
85
+
86
+
We can export them to nodes under label `Document` like this:
If a node label needs to appear as source or target of a relationship, but not exported as a node, you need to [declare](../core/flow_def#target-declarations) the label with necessary configuration.
125
+
126
+
The dataclass to describe the declaration is specific to each target (e.g. `cocoindex.targets.Neo4jDeclarations`),
127
+
while they share the following common fields:
128
+
129
+
*`nodes_label` (required): The label of the node.
130
+
* Options for [storage indexes](../core/flow_def#storage-indexes).
131
+
*`primary_key_fields` (required)
132
+
*`vector_indexes` (optional)
133
+
134
+
Continuing the same example above.
135
+
Considering we want to extract relationships from `Document` to `Place` later (i.e. a document mentions a place), but the `Place` label isn't exported as a node, we need to declare it:
136
+
137
+
```python
138
+
flow_builder.declare(
139
+
cocoindex.targets.Neo4jDeclarations(
140
+
connection=...,
141
+
nodes_label="Place",
142
+
primary_key_fields=["name"],
143
+
),
144
+
)
145
+
```
146
+
147
+
### Relationships to Export
148
+
149
+
Here's how CocoIndex data elements map to relationships in the graph:
150
+
151
+
| CocoIndex Element | Graph Element |
152
+
|-------------------|------------------|
153
+
| an export target | relationships with a unique type |
154
+
| a collected row | a relationship |
155
+
| a field | a property of relationship, or a property of source/target node, based on configuration |
156
+
157
+
Note that the type used in different `Relationships`s should be unique.
158
+
159
+
`cocoindex.targets.Relationships` is to describe mapping to relationships. It has the following fields:
160
+
161
+
*`rel_type` (`str`): The type of the relationship.
162
+
*`source`/`target` (`cocoindex.targets.NodeFromFields`): Specify how to extract source/target node information from specific fields in the collected row. It has the following fields:
163
+
*`label` (`str`): The label of the node.
164
+
*`fields` (`Sequence[cocoindex.targets.TargetFieldMapping]`): Specify field mappings from the collected rows to node properties, with the following fields:
165
+
*`source` (`str`): The name of the field in the collected row.
166
+
*`target` (`str`, optional): The name of the field to use as the node field. If unspecified, will use the same as `source`.
167
+
168
+
:::note Map necessary fields for nodes of relationships
169
+
170
+
You need to map the following fields for nodes of each relationship:
171
+
172
+
* Make sure all primary key fields for the label are mapped.
173
+
* Optionally, you can also map non-key fields. If you do so, please make sure all value fields are mapped.
174
+
175
+
:::
176
+
177
+
All fields in the collector that are not used in mappings for source or target node fields will be mapped to relationship properties.
178
+
179
+
For example, consider we have collected the following rows, to describe places mentioned in each file, along with embeddings of the places:
The `doc_filename` field is mapped to `Document.filename` property for the source node, while `place_name` and `place_embedding` are mapped to `Place.name` and `Place.embedding` properties for the target node.
218
+
The remaining field `location` becomes a property of the relationship.
219
+
For the data above, we get a bunch of relationships like this:
The nodes and relationships we got above are discrete elements.
281
+
To fit them into a connected property graph, CocoIndex will match and deduplicate nodes automatically:
282
+
283
+
* Match nodes based on their primary key values. Nodes with the same primary key values are considered as the same node.
284
+
* For non-primary key fields (a.k.a. value fields), CocoIndex will pick the values from an arbitrary one.
285
+
If multiple nodes (before deduplication) with the same primary key provide value fields, an arbitrary one will be picked.
286
+
287
+
:::note
288
+
289
+
The best practice is to make the value fields consistent across different appearances of the same node, to avoid non-determinism in the exported graph.
290
+
291
+
:::
292
+
293
+
After matching and deduplication, we get the final graph:
Copy file name to clipboardExpand all lines: docs/docs/targets/kuzu.md
+12-2Lines changed: 12 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,20 +3,23 @@ title: Kuzu
3
3
description: CocoIndex Kuzu Target
4
4
toc_max_heading_level: 4
5
5
---
6
+
import { ExampleButton } from '../../src/components/GitHubButton';
7
+
8
+
# Kuzu
6
9
7
10
Exports data to a [Kuzu](https://kuzu.com/) graph database.
8
11
9
12
## Get Started
10
13
11
-
Read [Property Graph Targets](../property-graph/index.md) for more information to get started on how it works.
14
+
Read [Property Graph Targets](./targets/index.md#property-graph-targets) for more information to get started on how it works in CocoIndex.
12
15
13
16
## Spec
14
17
15
18
CocoIndex supports talking to Kuzu through its [API server](https://github.com/kuzudb/api-server).
16
19
17
20
The `Kuzu` target spec takes the following fields:
18
21
19
-
*`connection` ([auth reference](../../core/flow_def#auth-registry) to `KuzuConnectionSpec`): The connection to the Kuzu database. `KuzuConnectionSpec` has the following fields:
22
+
*`connection` ([auth reference](../core/flow_def#auth-registry) to `KuzuConnectionSpec`): The connection to the Kuzu database. `KuzuConnectionSpec` has the following fields:
20
23
*`api_server_url` (`str`): The URL of the Kuzu API server, e.g. `http://localhost:8123`.
21
24
*`mapping` (`Nodes | Relationships`): The mapping from collected row to nodes or relationships of the graph. For either [nodes to export](#nodes-to-export) or [relationships to export](#relationships-to-export).
0 commit comments