Multiple vertices generated for the same index value and vertex properties missing with RF3

[sauverma]

Please include configurations and logs if available.

For confirmed bugs, please report:

• Version: 0.5.0
• Storage Backend: scylla enterprise 2020.1.6-0.20210202.c6a70535c
• Mixed Index Backend: none
• Link to discussed bug: https://lists.lfaidata.foundation/g/janusgraph-dev/message/1495
• Expected Behavior: The vertex and edge meta should be correctly stamped without enabling ConsistenyModifier.LOCK
• Current Behavior: Observation is that for many vertices with the same index value, multiple vertex IDs are generated as seen below
• Steps to reproduce
• Create a keyspace in scylla with RF3
• Load the janusgraph schema, the schema needs to have an index as mentioned in https://lists.lfaidata.foundation/g/janusgraph-users/message/5706
• Insert a few vertices via using multiple clients (each client using Embedded JanusGraph), and check if the vertices got ingested with correct metadata via gremlin console

[li-boxuan]

Looks like this is caused by a race condition. As suggested by https://docs.janusgraph.org/advanced-topics/eventual-consistency/, if you want to keep uniqueness constraint, you need to enable locking.

[sauverma]

hello @li-boxuan

The issue still persists as mentioned at after enabling Lock consistencyModifier (https://lists.lfaidata.foundation/g/janusgraph-users/topic/81197948?p=Created,,,20,1,0,0::recentpostdate/sticky,,,20,2,0,81197948)

Thanks

[li-boxuan]

Enabling locking does not resolve existing corrupted nodes. Do new nodes suffer the same problem after enabling locking?

[sauverma]

yes @li-boxuan we had truncated the keyspace started afresh after enabling locks

[sauverma]

2 types in which the index corruption is visible

• gremlin>g.V().has('id','131594d6a416666b401a9e48e54ebc8f22be75e2593c5d98e2d9ecfd719d5f29').has('type','email_sha256_lowercase').valueMap(true) ==>[dpts_678:[1595548800],label:vertex,id:201523209257056,id:[19df651e-90d5-47f6-af2e-35dcb59bcc0a],type:[id_mid_10],soft_del:[false],country_GBR:[678]]

• gremlin> g.V().has('id','y-teYNKkd1lw2pEUORQTqf3f6JPf19cIZsfg--').has('type','id_mid_24').valueMap() ==>[]

[porunov]

@sauverma Did you enable batch-loading for your transactions? I'm not sure if that could cause problems but enabling batch-loading automatically disables consistency checks. That said, not sure if that influences locks