You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/ROOT/pages/database-internals/concurrent-data-access.adoc
+46-36Lines changed: 46 additions & 36 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,41 +25,51 @@ All the anomalies listed here can only occur with the read-committed isolation l
25
25
In Cypher, it is possible to acquire write locks to simulate improved isolation in some cases.
26
26
Consider the case where multiple concurrent Cypher queries increment the value of a property.
27
27
Due to the limitations of the _read-committed isolation level_, the increments might not result in a deterministic final value.
28
-
If there is a direct dependency, Cypher automatically acquires a write lock before reading.
29
-
A direct dependency is when the right-hand side of `SET` has a dependent property read in the expression or the value of a key-value pair in a literal map.
30
28
31
-
For example, if you run the following query by one hundred concurrent clients, it is very likely not to increment the property `n.prop` to 100, unless a write lock is acquired before reading the property value.
32
-
This is because all queries read the value of `n.prop` within their own transaction, and cannot see the incremented value from any other transaction that has not yet been committed.
33
-
In the worst-case scenario, the final value would be as low as 1 if all threads perform the read before any has committed their transaction.
29
+
Cypher automatically acquires write locks in some cases, but not in others.
30
+
When a Cypher query uses the `SET` clause to update a property, it may or may not acquire a write lock on the node or relationship being updated, depending on whether there is a direct dependency on the property being read.
34
31
35
-
.Cypher can acquire a write lock
36
-
====
37
-
The following example requires a write lock, and Cypher automatically acquires one:
32
+
==== Acquiring a write lock automatically
38
33
34
+
When a Cypher query has a direct dependency on the property being read, Cypher automatically acquires a write lock before reading the property.
35
+
This is the case when the query uses the `SET` clause to update a property on a node or relationship, and the right-hand side of the `SET` clause has a dependency on the property being read.
36
+
For example, in the following queries, the right-hand side of `SET` has a dependent property read in an expression or a value of a key-value pair in a literal map.
37
+
38
+
.Incrementing a property using an expression
39
+
====
39
40
[source, cypher, role="noheader"]
40
41
----
41
42
MATCH (n:Example {id: 42})
42
43
SET n.prop = n.prop + 1
43
44
----
45
+
This query increments the property `n.prop` by 1.
46
+
In this case, Cypher automatically acquires a write lock on the node `n` before reading the value of `n.prop`.
47
+
This ensures that no other concurrent queries can modify the node `n` while this query is running, thus preventing lost updates.
44
48
====
45
49
46
-
.Cypher can acquire a write lock
50
+
.Incrementing a property using a map literal
47
51
====
48
-
This example also requires a write lock, and Cypher automatically acquires one:
49
-
50
52
[source, cypher, role="noheader"]
51
53
----
52
54
MATCH (n)
53
55
SET n += {prop: n.prop + 1}
54
56
----
57
+
58
+
This query also increments the property `n.prop` by 1, but it does so using a map literal.
59
+
In this case, Cypher also acquires a write lock on the node `n` before reading the value of `n.prop`.
55
60
====
56
61
57
-
Due to the complexity of determining such a dependency in the general case, Cypher does not cover any of the following example cases:
62
+
==== No direct dependency to acquire a write lock
58
63
59
-
.Complex Cypher
60
-
====
61
-
Variable depending on results from reading the property in an earlier statement:
64
+
When a query does not have a direct dependency on the property being read, Cypher does not automatically acquire a write lock.
65
+
This means if you run multiple concurrent queries that read and write the same property, it is possible to end up with lost updates by allowing other concurrent queries to modify the property value at the same time.
62
66
67
+
For example, if you run the following queries by one hundred concurrent clients, it is very likely not to increment the property `n.prop` to 100, unless a write lock is acquired before reading the property value.
68
+
This is because all queries read the value of `n.prop` within their own transaction, and cannot see the incremented value from any other transaction that has not yet been committed.
69
+
In the worst-case scenario, the final value would be as low as 1 if all threads perform the read before any has committed their transaction.
70
+
71
+
.Variable depending on results from reading the property in an earlier statement
72
+
====
63
73
[source, cypher, role="noheader"]
64
74
----
65
75
MATCH (n)
@@ -69,47 +79,45 @@ SET n.prop = k + 1
69
79
----
70
80
====
71
81
72
-
.Complex Cypher
82
+
.Circular dependency between properties read and written in the same query
73
83
====
74
-
Circular dependency between properties read and written in the same query:
75
-
76
84
[source, cypher, role="noheader"]
77
85
----
78
86
MATCH (n)
79
87
SET n += {propA: n.propB + 1, propB: n.propA + 1}
80
88
----
81
89
====
82
90
91
+
Workaround::
83
92
To ensure deterministic behavior also in the more complex cases, it is necessary to explicitly acquire a write lock on the node in question.
84
93
In Cypher there is no explicit support for this, but it is possible to work around this limitation by writing to a temporary property.
85
-
86
-
.Explicitly acquire a write lock
94
+
For example, the following query acquires a write lock for the node by writing to a *dummy* property (`n._dummy_`) before reading the requested value (`n.prop`).
95
+
When acquired, the write lock ensures that no other concurrent queries can modify the node until the transaction is committed or rolled back.
96
+
The dummy property is used only to acquire the write lock, therefore, it can be removed immediately after the lock is acquired.
97
+
+
98
+
.Dummy property to acquire a write lock
87
99
====
88
-
This example acquires a write lock for the node by writing to a dummy property before reading the requested value:
89
-
90
100
[source, cypher, role="noheader"]
91
101
----
92
102
MATCH (n:Example {id: 42})
93
-
SET n._LOCK_ = true
103
+
SET n._dummy_ = true
104
+
REMOVE n._dummy_
94
105
WITH n.prop AS p
95
106
// ... operations depending on p, producing k
96
107
SET n.prop = k + 1
97
-
REMOVE n._LOCK_
98
108
----
99
109
====
100
110
101
-
The existence of the `+SET n._LOCK_+` statement before the read of the `n.prop` read ensures the lock is acquired before the read action, and no updates are lost due to enforced serialization of all concurrent queries on that specific node.
102
-
103
111
=== Non-repeatable reads
104
112
105
113
A non-repeatable read is when the same transaction reads the same data but gets inconsistent results.
106
114
This can easily happen if reading the same data twice in a query and the data gets modified in-between by another concurrent query.
107
115
108
-
.Non-repeatable read
109
-
====
110
-
The following example query shows that reading the same property twice can give inconsistent results.
116
+
For example, the following query shows that reading the same property twice can give inconsistent results.
111
117
If there are other queries running concurrently, it is not guaranteed that `p1` and `p2` have the same value.
112
118
119
+
.Non-repeatable read
120
+
====
113
121
[source, cypher, role="noheader"]
114
122
----
115
123
MATCH (n:Example {id: 42})
@@ -132,17 +140,17 @@ Similarly, the entity may not appear at all if the property is changed to a prev
132
140
133
141
This anomaly can only occur with operators that scan an index, or parts of an index, for example link:{neo4j-docs-base-uri}/cypher-manual/current/planning-and-tuning/operators/operators-detail/#query-plan-node-index-scan[`NodeIndexScan`] or link:{neo4j-docs-base-uri}/cypher-manual/current/planning-and-tuning/operators/operators-detail/#query-plan-directed-relationship-index-seek-by-range[`DirectedRelationshipIndexSeekByRange`].
134
142
135
-
.Missing and double read
136
-
====
137
143
In the following query, each node `n` that has the property `prop` is expected to appear exactly once.
138
144
However, concurrent updates that modify the `prop` property during index scanning may cause a node to appear multiple times or not at all in the result set.
145
+
146
+
.Missing and double read
147
+
====
139
148
[source, cypher, role="noheader"]
140
149
----
141
150
MATCH (n:Example) WHERE n.prop IS NOT NULL
142
151
RETURN n
143
152
----
144
153
====
145
-
146
154
== Locks
147
155
148
156
When a write transaction occurs, Neo4j takes locks to preserve data consistency while updating.
@@ -279,15 +287,13 @@ Setting `db.lock.acquisition.timeout` to `0` -- which is the default value -- di
279
287
280
288
This feature cannot be set dynamically.
281
289
282
-
.Configure lock acquisition timeout
290
+
.Set the timeout to ten seconds
283
291
====
284
-
Set the timeout to ten seconds.
285
292
[source, parameters]
286
293
----
287
294
db.lock.acquisition.timeout=10s
288
295
----
289
296
====
290
-
291
297
[[deadlocks]]
292
298
== Deadlocks
293
299
@@ -319,6 +325,7 @@ Other code that requires synchronization should be synchronized in such a way th
319
325
For example, running the following two queries in https://neo4j.com/docs/operations-manual/current/tools/cypher-shell/[Cypher-shell] at the same time will result in a deadlock because they are attempting to modify the same node properties concurrently:
320
326
321
327
.Transaction A
328
+
====
322
329
[source, cypher, indent=0, role=nocopy noplay]
323
330
----
324
331
:begin
@@ -327,8 +334,9 @@ WITH collect(n) as nodes
327
334
CALL apoc.util.sleep(5000)
328
335
MATCH (m:Test2) SET m.prop = 1;
329
336
----
330
-
337
+
====
331
338
.Transaction B
339
+
====
332
340
[source, cypher, indent=0, role=nocopy noplay]
333
341
----
334
342
:begin
@@ -347,6 +355,8 @@ The transaction will be rolled back and terminated. Error: ForsetiClient[transac
347
355
Client[6697] waits for [ForsetiClient[transactionId=6698, clientId=1]]]
348
356
----
349
357
358
+
====
359
+
350
360
[NOTE]
351
361
====
352
362
The Cypher clause `MERGE` takes locks out of order to ensure the uniqueness of the data, and this may prevent Neo4j's internal sorting operations from ordering transactions in a way that avoids deadlocks.
0 commit comments