Support walks, trails, paths; shortest and cheapest paths matching

Author: boggle

cip/1.accepted/CIP2017-01-18-isomorphic-matching.adoc

@@ -0,0 +1,214 @@
+= CIP2017-01-18 - Configurable Pattern Matching Semantics
+:numbered:
+:toc:
+:toc-placement: macro
+:source-highlighter: codemirror
+
+*Author:* Stefan Plantikow <stefan.plantikow@neotechnology.com>
+
+This proposal is a response to CIR-2017-174.
+
+== Motivation
+
+Currently Cypher uses pattern matching semantics that treats all patterns that occur in a `MATCH` clause as a unit (called a *uniqueness scope*) and only considers pattern instances that bind different relationships to each simple relationship pattern variable and to each element of a variable length relationship pattern variable.
+This has come to be called *cypermorphism* informally and is a variation of edge isomorphism.
+
+Cyphermorphism lies at the intersection of returning as many results as possible while still ruling out returning an infinite number of paths when matching graphs that contain cycles.
+
+However, the notion of *uniqueness scope* has proven to be non-standard and occasionally confusing for users and cyphermorphic matching is not tractable in terms of computational complexity for some graphs.
+
+The CIP aims to address these issues.
+
+== Background
+
+Each pattern consists of a comma separated list of *pattern parts*.
+Pattern parts are bound to a variable and consist of a linear chain of connected node and relationship patterns.
+
+Note that while Cypher allows omitting path, node, and relationship variables in a pattern this should only be considered as syntactic sugar, i.e. all parts of a pattern are always bound to a variable name from the viewpoint of pattern matching semantics (names are either provided in the query or automatically generated by a conforming implementation).
+
+== Proposal
+
+This CIP proposes to replace the notion of *uniqueness scope* and *cyphermorphism* and all associated rules by providing new, configurable pattern matching semantics for Cypher as outlined in this section.
+
+=== Rename PATH type
+
+This CIP proposes to rename the cypher type `PATH` to `WALK`.
+
+=== Definitions
+
+This CIP introduces the following kinds of walks:
+
+* `WALK`: A walk is an arbitrary, non-empty sequence of alternating nodes and relationships that starts with a node and ends with a node.
+* `TRAIL`: A trail is a walk that does not contain the same relationship twice.
+* `PATH`: A simple path is a trail that does not contain the same node twice unless that node is both the start node and the end node of the path.
+
+Note that every `PATH` is a `TRAIL` and that every `TRAIL` is a `WALK`.
+
+=== Pattern binder type
+
+This CIP proposes to name the variable of a pattern element of a pattern part to *pattern binder* in the grammar.
+Note that a pattern binder is always bound to a linear sequence of patterns of its pattern element.
+
+This CIP proposes introducing the notion of a *pattern binder type* that may be writtern before a pattern binder in a read-only pattern (i.e. a pattern that is not used as an argument to an updating clause) and limits the set of valid pattern instances that are considered as potential matches for the following pattern element:
+
+* `WALK` This pattern binder should only be bound to a `WALK` that matches all node, relationship, and path patterns in the following pattern element
+* `TRAIL` This pattern binder should only be bound to a `TRAIL` that matches all node, relationship, and path patterns in the following pattern element
+* `PATH` This pattern binder should only be bound to a simple `PATH` that matches all node, relationship, and path patterns in the following pattern element
+
+The pattern binder type may be futher qualified with one of the following prefixes:
+
+* `OPEN WALK|TRAIL|PATH` This pattern binder should only be bound to walks (or trails, or paths respectively) whose start and end nodes are not the same node
+* `CLOSED WALK|TRAIL|PATH` This pattern binder should only be bound to walks (or trails, or paths respectively) whose start and end nodes are the same node
+
+The following additional pattern binder types are proposed to accomodate existing terminology that is commonly used in graph theory:
+
+* `CIRCUIT` is a synonym for `CLOSED TRAIL`
+* `CYCLE` is a synonym for `CLOSED PATH`
+
+Implementations are advised to signal a warning for every use of an `OPEN` pattern binder type if the two endpoints of the pattern element are both unbound and both use the same variable name.
+
+Implementations are advised to signal a warning for every use of an `CLOSED` pattern binder type if the two endpoints of the pattern element are both unbound and both use a different variable name.
+
+=== Pattern matching mode
+
+This CIP proposes introducing the notion of a *pattern matching mode* that may be writtern as a prefix to a read-only pattern (i.e. a pattern that is not used as an argument to an updating clause) and applies to all pattern parts in that pattern.
+
+=== MATCH EVERY mode
+
+This CIP proposes the new `MATCH EVERY [WALK|TRAIL|PATH]` pattern matching mode that matches every walk (or trail, or path respectively) as described by all node, relationship, and path patterns in the following pattern element.
+This may return an infinite or at least a very large result for some graphs.
+
+Implementations are advised to signal a warning for every use of `MATCH EVERY (OPEN|CLOSED) WALK` that may lead to the generation of an infinite result set.
+
+=== MATCH SHORTEST mode
+
+This CIP proposes the new `MATCH SHORTEST [WALK|TRAIL|PATH]` pattern matching mode that matches every _shortest_ walks (or trails, or paths respectively) as described by all node, relationship, and path patterns in the following pattern element.
+
+This CIP proposes to deprecate the existing syntax for both `shortestPath` and `allShortestPaths` matching of Cypher.
+
+=== Weighting rules
+
+This CIP proposes that pattern elements may be further suffixed with a weighting rule of one of the following forms:
+
+* `WEIGHT r IN <aggregation> AS <weight>` Calculates a weight `<weight>` by evaluating the given `<aggregation>` for each relationship `r` in the currently matched walk
+* `WEIGHT |<expr>| AS <weight>` Calculates a weight `<weight>` by summing the results of evaluating `abs(<expr>)` for each relationship `r` in the currenlty matched walk in a special scope that only contains all properties of `r` as variables
+
+=== Product function
+
+To support a common family of weight calculations, this CIP proposes the introduction of a new aggregate function `product` for computing the product of a set of numbers.
+
+=== MATCH CHEAPEST mode
+
+This CIP proposes the new `MATCH CHEAPEST [WALK|TRAIL|PATH]` pattern matching mode that matches every cheapest walk (or trail, or path respectively) as described by all node, relationship, and path patterns in the following pattern element and its concluding mandatory weighting rule that is prefixed with `BY`.
+
+The mandatory weighting rule may omit specifying an alias name for the computed weight and it's aggregation must be monotone (i.e. the sequence of intermediary results obtained by computing the aggregation incrementally over all input values in any order is always monotonically increasing).
+
+A conforming implementation is expected to raise a runtime error when the monotonicity of a monotone aggregation is violated at runtime.
+
+A conforming implementation may raise a compile time error when it can statically prove that the monotonicity of a monotone aggregation may be violated at runtime.
+
+=== Nondeterministic matching modes
+
+This CIP proposed using the keywords `SINGLE` and `FIRST` to modify pattern matching modes to return exactly zero or one arbitrarily chosen match.
+
+The supported forms are:
+
+* `MATCH SINGLE SHORTEST [WALK|TRAIL|PATH]` to return at most a single shortest walk (or trail, or path respectively)
+* `MATCH SINGLE CHEAPEST [WALK|TRAIL|PATH]` to return at most a single cheapest walk (or trail, or path respectively)
+* `MATCH FIRST [WALK|TRAIL|PATH]` to return at most a single arbitrary walk (pendent to `MATCH EVERY`)
+* `MATCH SINGLE [WALK|TRAIL|PATH]` to return at most a single walk (or trail, or path respectively) using default pattern matching semantics (defined below)
+
+=== Default pattern matching semantics
+
+It is proposed that a conforming implementation should provide a pre-parser option for defining the default pattern binder type for each pattern matching mode as well as the default pattern matching mode:
+
+* `match-every=walk|trail|path` for configuring the default pattern binder type for each use of the `MATCH EVERY` pattern matching mode
+* `match-shortest=walk|trail|path` for configuring the default pattern binder type for each use of the `MATCH SHORTEST` pattern matching mode
+* `match-cheapest=walk|trail|path` for configuring the default pattern binder type for each use of the `MATCH CHEAPEST` pattern matching mode
+* `match=every|shortest` for configuring the default pattern matching mode
+
+Using these pre-parser options, current Cypher pattern matching semantics closely correspond to `match-every=trail`, `match-shortest=trail`, `match=every` (except for the use of the uniqueness scope)
+
+This CIP proposes to change Cypher's default pattern matching semantics to `match-every=trail`, `match-shortest=walk`, `match-cheapest=walk`, `match=shortest`.
+
+=== Variable length patterns
+
+This CIP aligns with the introduction of path patterns by proposing that variable length patterns are to be deprecated in favor of path patterns.
+
+To simplify this migration and deprecation, this CIP proposes that any pattern element that contains a variable length pattern but no path pattern should match a `TRAIL` be default.
+
+=== Path predicates
+
+This CIP further proposes to introduce additional predicates and functions for working with walks
+
+* `open(p)`: true if the start and the end node of `p` are not the same node
+* `closed(p)`: true if the start and the end node of `p` are the same node
+* `trail(p)`: `p` if `p` contains no duplicate relationships, `NULL` otherwise
+* `path(p)`: `p` if `p` contains no duplicate relationships and either no duplicate nodes at all or the start node and the end node are the same node, `NULL` otherwise
+* `circuit(p)`:  `trail(p)`, if `closed(p)` is true, `NULL` otherwise
+* `cycle(p)`: `path(p)`, if `closed(p)` is true, `NULL` otherwise
+* `overlap(p1, p2)`: the shared walk between `p1` and `p2`, or `NULL` otherwise
+* `overlap(nodes(p1), nodes(p2))`: the shared walk between `nodes(p1)` and `nodes(p2)`, or `NULL` otherwise
+* `overlap(rels(p1), rels(p2))`: the shared walk between `rels(p1)` and `rels(p2)`, or `NULL` otherwise
+* `adjacent(p1, p2)`: true if `startNode(p1) IN [startNode(p2), endNode(p2)]` or `endNode(p1) IN [startNode(p2), endNode(p2)]`
+* `adjacent(nodes(p1), nodes(p2))`: true if `startNode(p1) IN [startNode(p2), endNode(p2)]` or `endNode(p1) IN [startNode(p2), endNode(p2)]`
+* `adjacent(rels(p1), rels(p2))`: true if `startNode(p1) IN [startNode(p2), endNode(p2)]` or `endNode(p1) IN [startNode(p2), endNode(p2)]`
+* `adjacent(r1, r2)`: true if `startNode(r1) IN [startNode(r2), endNode(r2)]` or `endNode(r1) IN [startNode(r2), endNode(r2)]`
+* `adjacent(n1, n2)`: true if `EXISTS (n1)-[]-(n2)`
+
+== Examples
+
+The following examples demonstrates various ways in which the newly proposed constructs may be used if this CIP is adopted.
+
+=== Matching shortest paths
+
+[source=cypher]
+----
+// shortestPath(...) today becomes:
+MATCH SHORTEST TRAIL p=(a)-[r*]->(b)
+RETURN * LIMIT 1
+
+// allShortestPath(...) today becomes:
+MATCH SHORTEST TRAIL p=(a)-[r*]->(b)
+RETURN p
+----
+
+=== Matching cheapest paths
+
+[source=cypher]
+----
+MATCH CHEAPEST PATH p=(a)-/(:LOVES|:LIKES)*/->(b) BY WEIGHT |strength| AS w
+RETURN p AS paht, w AS weight
+----
+
+=== Matching with existing semantics
+
+`overlap` may be used to express Cypher's current pattern matching semantics.
+
+[source=cypher]
+----
+// Today (using same uniqueness scope for pat1, pat2, and pat)
+MATCH pat1=..., pat2=..., pat3=...
+
+// This CIP
+MATCH EVERY pat1=...
+MATCH EVERY pat2=... WHERE length(overlap(pat1, pat2)) > 1
+MATCH EVERY pat3=...
+WHERE
+    length(overlap(pat1, pat3))>1 OR
+    length(overlap(pat2, pat3))>1
+)
+----
+
+== Benefits to this proposal
+
+This proposal adds a generic facility to Cypher for expressing desired pattern matching semantics.
+
+== Caveats to this proposal
+
+A moderate increase in language complexity.
+
+A substantial departure from current pattern matching semantics.
+However, care has been taken to retain access to current semantics.
+
+`MATCH EVERY (OPEN|CLOSED) WALK` allows for non-terminating queries.