Open
Conversation
* update submodule * update submodule * update clam-core post-merge * implement efficient skip for StateIndex iterator * update submodule * make arc_into_iter efficient as well * add support for returning index to node list * concrete type for index iterator * node scan no longer eagerly filters and collects the node ids * update submodule
* update clam-core * add cargo lock
…edge` and graph properties (#2493) * Add logging and replay for add_updates for edges * Remove separate add_edge_properties wal entry We can reuse existing wal code for add_edge * Use set_or_unify_id_and_dtype during replay * Add logging and replay for AddEdgeMetadata * chore: apply tidy-public auto-fixes * Remove src_id and dst_id for add_metadata * Use AddEdgeMetadata logging methods * Add set_lsn method for writers * Use correct mapper for metadata * Create add_metadata_impl and reuse * chore: apply tidy-public auto-fixes * Add wal logging for NodeView.add_updates * Create add_metadata_impl for NodeView and reuse * Add wal logging and replay for NodeView.add_metadata * chore: apply tidy-public auto-fixes * Add wal for set_node_type * Add FIXME for actual set_node_type logging * Add logging and replay for graph_props * Implement set_lsn for GraphPropWriter * Return GraphPropWriterT * Call wal.log in add_properties * Call wal.log in add_properties * Remove some unwraps and collects in replay * Fix some more merge conflicts * Add logging for set_node_type * Run fmt * Run fmt * Cleanup and fix reserve_and_lock_segment * Remove Option from WriteLockedGraphPropPages writer * Check lsn during graph prop replay * Refactor graph props add_metadata * Implement logging and replay for graph metadata * Use is_update correctly * Implement logging and replay for delete_edge * Run fmt * Simplify some comments * Run fmt * fix some review comments * fmt * Add missing ? on unify_types * Update comment * Add new nodes into resolver in replay_delete_edge * Fix src, dst and edge writes in replay_delete_edge * Use STATIC_GRAPH_LAYER_ID * Add wal logging for EdgeView.delete_edge * Run fmt * Cleanup some config --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Fabian Murariu <murariu.fabian@gmail.com> # Conflicts: # db4-storage/src/pages/edge_page/writer.rs # raphtory/src/db/api/storage/storage.rs
# Conflicts: # raphtory/src/db/api/storage/storage.rs
fabianmurariu
force-pushed
the
db_v4_layer_optim
branch
from
312611c to
c4f32bc
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
| Benchmark suite | Current: c2a4632 | Previous: 4fe205a | Ratio |
|---|---|---|---|
large/1k random edge additions |
1323550 ns/iter (± 133380) |
580946 ns/iter (± 48661) |
2.28 |
large/1k random edge additions with numeric string input |
1767745 ns/iter (± 118359) |
751536 ns/iter (± 69189) |
2.35 |
lotr_graph/num_edges |
3 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph/num_nodes |
352 ns/iter (± 1) |
1 ns/iter (± 0) |
352 |
lotr_graph/iterate nodes |
4822 ns/iter (± 8) |
1846 ns/iter (± 10) |
2.61 |
lotr_graph/graph_latest |
2 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
lotr_graph_materialise/materialize |
10242949 ns/iter (± 198117) |
1556941 ns/iter (± 4096) |
6.58 |
lotr_graph_window_100/num_edges |
25 ns/iter (± 0) |
8 ns/iter (± 0) |
3.13 |
lotr_graph_window_100/num_nodes |
368 ns/iter (± 2) |
5 ns/iter (± 0) |
73.60 |
lotr_graph_window_100_materialise/materialize |
10278784 ns/iter (± 100773) |
1715006 ns/iter (± 6924) |
5.99 |
lotr_graph_window_10/has_node_existing |
127 ns/iter (± 9) |
62 ns/iter (± 12) |
2.05 |
lotr_graph_window_10/iterate nodes |
30619 ns/iter (± 114) |
11873 ns/iter (± 207) |
2.58 |
lotr_graph_window_10_materialise/materialize |
6770903 ns/iter (± 110953) |
1040130 ns/iter (± 3339) |
6.51 |
lotr_graph_subgraph_10pc/has_edge_existing |
214 ns/iter (± 5) |
93 ns/iter (± 1) |
2.30 |
lotr_graph_subgraph_10pc/num_nodes |
359 ns/iter (± 1) |
4 ns/iter (± 0) |
89.75 |
lotr_graph_subgraph_10pc/has_node_existing |
92 ns/iter (± 2) |
36 ns/iter (± 0) |
2.56 |
lotr_graph_subgraph_10pc/iterate nodes |
3596 ns/iter (± 16) |
859 ns/iter (± 2) |
4.19 |
lotr_graph_subgraph_10pc_windowed/iterate nodes |
4987 ns/iter (± 11) |
1389 ns/iter (± 8) |
3.59 |
lotr_graph_window_50_layered/num_nodes |
65291 ns/iter (± 4701) |
27372 ns/iter (± 839) |
2.39 |
lotr_graph_window_50_layered/has_node_existing |
824 ns/iter (± 66) |
128 ns/iter (± 13) |
6.44 |
lotr_graph_window_50_layered/max_id |
75116 ns/iter (± 2410) |
31699 ns/iter (± 2846) |
2.37 |
lotr_graph_window_50_layered/iterate nodes |
114740 ns/iter (± 4739) |
20399 ns/iter (± 144) |
5.62 |
lotr_graph_window_50_layered/iterate edges |
181119 ns/iter (± 287) |
87369 ns/iter (± 809) |
2.07 |
lotr_graph_window_50_layered/graph_latest |
105974 ns/iter (± 1232) |
43211 ns/iter (± 1166) |
2.45 |
lotr_graph_window_50_layered_materialise/materialize |
25348599 ns/iter (± 569941) |
3434339 ns/iter (± 18046) |
7.38 |
lotr_graph_persistent_window_50_layered/num_edges_temporal |
622315 ns/iter (± 8468) |
202616 ns/iter (± 4322) |
3.07 |
lotr_graph_persistent_window_50_layered/num_nodes |
82981 ns/iter (± 3703) |
37210 ns/iter (± 4010) |
2.23 |
lotr_graph_persistent_window_50_layered/has_node_existing |
1039 ns/iter (± 162) |
181 ns/iter (± 80) |
5.74 |
lotr_graph_persistent_window_50_layered/max_id |
95532 ns/iter (± 3489) |
44411 ns/iter (± 809) |
2.15 |
lotr_graph_persistent_window_50_layered/iterate nodes |
156869 ns/iter (± 293) |
36772 ns/iter (± 212) |
4.27 |
lotr_graph_persistent_window_50_layered/graph_latest |
163501 ns/iter (± 6931) |
63508 ns/iter (± 2678) |
2.57 |
lotr_graph_persistent_window_50_layered_materialise/materialize |
55240198 ns/iter (± 320444) |
5185393 ns/iter (± 30547) |
10.65 |
lotr_graph/proto_decode |
8296184 ns/iter (± 67046) |
1892290 ns/iter (± 21928) |
4.38 |
lotr_graph/proto_encode |
6552316 ns/iter (± 70135) |
1153276 ns/iter (± 41101) |
5.68 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Optimise layers to actually use the available indexes in v4
Why are the changes needed?
Does this PR introduce any user-facing change? If yes is this documented?
How was this patch tested?
Are there any further changes required?