Related::Relationship can become semi-inconsistent

[zond]

Of course the underlying data structures don't become inconsistent for realz, since Redis prevents that. But since the #Related.redis.multi do lines in Relationship are commented out (I suppose to enable use of Distributed Redis connections, and with good reason) we can get relationships that are broken in the sense of being only one direction, for example.

Perhaps some kind of self healing would be a useful solution: Every time (or only some times) when an edge is traversed, Related checks the consistency of the edge (that it exists in all ordered and non-ordered sets it should), and fixes it if it is not.

Solving this kind of thing without distributed transactions is of course hard, but this kind of self healing is at least somewhat tried and true.

[niho]

Yes, I agree that this can be a problem if the consistency of the graph is important. It's hard to predict how much of a problem it is in reality though. The time window for inconsistencies to occur is extremely small, since the operations are supposed to be sent and executed all at the same time in parallel (I'm unsure if they are right now though).

Self healing might be a possible solution. But it might conflict somewhat with the stated goal of Related to provide reliable and consistent low latency performance though.

Another possible solution could be to instead repair the graph in a completely separate background job. That wouldn't affect the request latency and wouldn't add any extra complexity to the core of Related, since the code for that can be kept completely separate (like a script in a contrib/ folder for example).

I imagine a ./contrib/repair script that walks the graph and repairs any inconsistencies that it finds. It can be run as a simple cron job for example.