Can a Graph Index be used in the middle of traversal?

[roi-tevel]

Hi,
Assuming we have such traversal:
g.V().has('first_indexed_prop', 'first_value').out('some_edge').out('some_other_edge').has('other_indexed_prop','other_value').out().count().next()
will JanusGraph be able to use the index we have on other_indexed_prop of the Vertex? (when I run it with profile() it seems as if it doesn't use it, even when its a mixed index)
or do we need to add other_indexed_prop as a property of the edge (some_other_edge) and add an edge index in order for JG to be able to use another index in the middle of a traversal?
Thanks,
Roi

[vtslab]

This is discussed in https://docs.janusgraph.org/schema/index-management/index-performance/#using-vertex-centric-indexes-on-adjacent-vertex-ids

Retrieving vertices further in a traversal before filtering is not very expensive (compared to performing a full table scan), unless your graph has lots of "other edges". In the latter case, the link above provides a solution, at the cost of some additional code.

[roi-tevel]

Hey, thanks, from the profile it seems that it is doing TraversalFilterStep in case of where, and that this is somehow much much more efficient than the HasStep step, but I am not sure why?

this is with has:

==>Traversal Metrics
Step                                                               Count  Traversers       Time (ms)    % Dur
=============================================================================================================
JanusGraphStep([],[search_name.eq(amazon)])                            1           1          86.283     0.25
  constructGraphCentricQuery                                                                   0.048
  GraphCentricQuery                                                                           86.098
    \_condition=(search_name = amazon)
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=[(search_name = amazon)]:bySearchName
    \_index=bySearchName
    \_index_impl=search
    backend-query                                                      1                      86.026
    \_query=bySearchName:[(search_name = amazon)]:bySearchName
JanusGraphMultiQueryStep                                               1           1           0.039     0.00
JanusGraphVertexStep(OUT,[subject],vertex)                             7           7          10.495     0.03
    \_condition=type[subject]
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=subject:SliceQuery[0x701AC0,0x701AC1)
    \_multi=true
    \_vertices=1
  optimization                                                                                 0.058
  backend-query                                                        7                      10.293
    \_query=subject:SliceQuery[0x701AC0,0x701AC1)
JanusGraphMultiQueryStep                                               7           7           0.047     0.00
JanusGraphVertexStep(OUT,[processed_by],vertex)                     8180        8180          44.067     0.13
    \_condition=type[processed_by]
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=processed_by:SliceQuery[0x701DE0,0x701DE1)
    \_multi=true
    \_vertices=7
  optimization                                                                                 0.051
  backend-query                                                     8180                      30.742
    \_query=processed_by:SliceQuery[0x701DE0,0x701DE1)
NoOpBarrierStep(2500)                                               8180        8180          22.035     0.06
HasStep([latest.eq(true)])                                                                 33942.107    99.52
CountGlobalStep                                                        1           1           0.041     0.00
                                            >TOTAL                     -           -       34105.117        -

this is without has:

==>Traversal Metrics
Step                                                               Count  Traversers       Time (ms)    % Dur
=============================================================================================================
JanusGraphStep([],[search_name.eq(amazon)])                            1           1           0.232     0.48
  constructGraphCentricQuery                                                                   0.069
  GraphCentricQuery                                                                            0.097
    \_condition=(search_name = amazon)
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=[(search_name = amazon)]:bySearchName
    \_index=bySearchName
    \_index_impl=search
JanusGraphMultiQueryStep                                               1           1           0.031     0.06
JanusGraphVertexStep(OUT,[subject],vertex)                             7           7           0.102     0.21
    \_condition=type[subject]
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=subject:SliceQuery[0x701AC0,0x701AC1)
    \_multi=true
    \_vertices=1
  optimization                                                                                 0.026
JanusGraphMultiQueryStep                                               7           7           0.024     0.05
JanusGraphVertexStep(OUT,[processed_by],vertex)                     8180        8180           4.701     9.63
    \_condition=type[processed_by]
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=processed_by:SliceQuery[0x701DE0,0x701DE1)
    \_multi=true
    \_vertices=7
  optimization                                                                                 0.005
JanusGraphMultiQueryStep                                            8180        8180           6.187    12.67
NoOpBarrierStep(2500)                                               8180        8180           4.803     9.84
TraversalFilterStep([JanusGraphPropertiesStep([...                    55          55          32.597    66.77
  JanusGraphPropertiesStep([latest],value)                            55          55          22.991
    \_condition=type[latest]
    \_orders=[]
    \_isFitted=true
    \_isOrdered=true
    \_query=latest:SliceQuery[0x5A80,0x5A81)
    \_multi=true
    \_vertices=8180
    optimization                                                                               0.024
  IsStep(eq(true))                                                                             3.724
CountGlobalStep                                                        1           1           0.142     0.29
                                            >TOTAL                     -           -          48.823        -

[roi-tevel]

This is discussed in https://docs.janusgraph.org/schema/index-management/index-performance/#using-vertex-centric-indexes-on-adjacent-vertex-ids

Retrieving vertices further in a traversal before filtering is not very expensive (compared to performing a full table scan), unless your graph has lots of "other edges". In the latter case, the link above provides a solution, at the cost of some additional code.

Thanks again, this also applies for using an index for ordering? Janus can not use the index for ordering if the order().by() is further in the traversal? it there a way around that?

[li-boxuan]

@roi-tevel The performance improvement you saw is due to multithreading. In the second profile, you can see \_multi=true in JanusGraphPropertiesStep, indicating that it uses multi-threading. Theoretically, we could (and we should) leverage multi-threading for the HasStep in the first query as well, but that is something not done yet (created #3244). Btw what JanusGraph version you are using?

[roi-tevel]

@li-boxuan Thanks, good to know. we are using version 0.6.0

[vtslab]

Janus can not use the index for ordering if the order().by() is further in the traversal? it there a way around that?

See: https://docs.janusgraph.org/v0.4/index-management/index-performance/#ordering
For mixed indices the order().by() modifier is pushed down to the indexing backend. This should work for successive has().order().by() steps, I am not sure what happens if the order().by() does not immediately follow the has() step.