1
+ .. _rcu_dereference_doc :
2
+
1
3
PROPER CARE AND FEEDING OF RETURN VALUES FROM rcu_dereference()
4
+ ===============================================================
2
5
3
6
Most of the time, you can use values from rcu_dereference() or one of
4
7
the similar primitives without worries. Dereferencing (prefix "*"),
@@ -8,7 +11,7 @@ subtraction of constants, and casts all work quite naturally and safely.
8
11
It is nevertheless possible to get into trouble with other operations.
9
12
Follow these rules to keep your RCU code working properly:
10
13
11
- o You must use one of the rcu_dereference() family of primitives
14
+ - You must use one of the rcu_dereference() family of primitives
12
15
to load an RCU-protected pointer, otherwise CONFIG_PROVE_RCU
13
16
will complain. Worse yet, your code can see random memory-corruption
14
17
bugs due to games that compilers and DEC Alpha can play.
@@ -25,24 +28,24 @@ o You must use one of the rcu_dereference() family of primitives
25
28
for an example where the compiler can in fact deduce the exact
26
29
value of the pointer, and thus cause misordering.
27
30
28
- o You are only permitted to use rcu_dereference on pointer values.
31
+ - You are only permitted to use rcu_dereference on pointer values.
29
32
The compiler simply knows too much about integral values to
30
33
trust it to carry dependencies through integer operations.
31
34
There are a very few exceptions, namely that you can temporarily
32
35
cast the pointer to uintptr_t in order to:
33
36
34
- o Set bits and clear bits down in the must-be-zero low-order
37
+ - Set bits and clear bits down in the must-be-zero low-order
35
38
bits of that pointer. This clearly means that the pointer
36
39
must have alignment constraints, for example, this does
37
40
-not- work in general for char* pointers.
38
41
39
- o XOR bits to translate pointers, as is done in some
42
+ - XOR bits to translate pointers, as is done in some
40
43
classic buddy-allocator algorithms.
41
44
42
45
It is important to cast the value back to pointer before
43
46
doing much of anything else with it.
44
47
45
- o Avoid cancellation when using the "+" and "-" infix arithmetic
48
+ - Avoid cancellation when using the "+" and "-" infix arithmetic
46
49
operators. For example, for a given variable "x", avoid
47
50
"(x-(uintptr_t)x)" for char* pointers. The compiler is within its
48
51
rights to substitute zero for this sort of expression, so that
@@ -54,16 +57,16 @@ o Avoid cancellation when using the "+" and "-" infix arithmetic
54
57
"p+a-b" is safe because its value still necessarily depends on
55
58
the rcu_dereference(), thus maintaining proper ordering.
56
59
57
- o If you are using RCU to protect JITed functions, so that the
60
+ - If you are using RCU to protect JITed functions, so that the
58
61
"()" function-invocation operator is applied to a value obtained
59
62
(directly or indirectly) from rcu_dereference(), you may need to
60
63
interact directly with the hardware to flush instruction caches.
61
64
This issue arises on some systems when a newly JITed function is
62
65
using the same memory that was used by an earlier JITed function.
63
66
64
- o Do not use the results from relational operators ("==", "!=",
67
+ - Do not use the results from relational operators ("==", "!=",
65
68
">", ">=", "<", or "<=") when dereferencing. For example,
66
- the following (quite strange) code is buggy:
69
+ the following (quite strange) code is buggy::
67
70
68
71
int *p;
69
72
int *q;
@@ -81,19 +84,19 @@ o Do not use the results from relational operators ("==", "!=",
81
84
after such branches, but can speculate loads, which can again
82
85
result in misordering bugs.
83
86
84
- o Be very careful about comparing pointers obtained from
87
+ - Be very careful about comparing pointers obtained from
85
88
rcu_dereference() against non-NULL values. As Linus Torvalds
86
89
explained, if the two pointers are equal, the compiler could
87
90
substitute the pointer you are comparing against for the pointer
88
- obtained from rcu_dereference(). For example:
91
+ obtained from rcu_dereference(). For example::
89
92
90
93
p = rcu_dereference(gp);
91
94
if (p == &default_struct)
92
95
do_default(p->a);
93
96
94
97
Because the compiler now knows that the value of "p" is exactly
95
98
the address of the variable "default_struct", it is free to
96
- transform this code into the following:
99
+ transform this code into the following::
97
100
98
101
p = rcu_dereference(gp);
99
102
if (p == &default_struct)
@@ -105,14 +108,14 @@ o Be very careful about comparing pointers obtained from
105
108
106
109
However, comparisons are OK in the following cases:
107
110
108
- o The comparison was against the NULL pointer. If the
111
+ - The comparison was against the NULL pointer. If the
109
112
compiler knows that the pointer is NULL, you had better
110
113
not be dereferencing it anyway. If the comparison is
111
114
non-equal, the compiler is none the wiser. Therefore,
112
115
it is safe to compare pointers from rcu_dereference()
113
116
against NULL pointers.
114
117
115
- o The pointer is never dereferenced after being compared.
118
+ - The pointer is never dereferenced after being compared.
116
119
Since there are no subsequent dereferences, the compiler
117
120
cannot use anything it learned from the comparison
118
121
to reorder the non-existent subsequent dereferences.
@@ -124,31 +127,31 @@ o Be very careful about comparing pointers obtained from
124
127
dereferenced, rcu_access_pointer() should be used in place
125
128
of rcu_dereference().
126
129
127
- o The comparison is against a pointer that references memory
130
+ - The comparison is against a pointer that references memory
128
131
that was initialized "a long time ago." The reason
129
132
this is safe is that even if misordering occurs, the
130
133
misordering will not affect the accesses that follow
131
134
the comparison. So exactly how long ago is "a long
132
135
time ago"? Here are some possibilities:
133
136
134
- o Compile time.
137
+ - Compile time.
135
138
136
- o Boot time.
139
+ - Boot time.
137
140
138
- o Module-init time for module code.
141
+ - Module-init time for module code.
139
142
140
- o Prior to kthread creation for kthread code.
143
+ - Prior to kthread creation for kthread code.
141
144
142
- o During some prior acquisition of the lock that
145
+ - During some prior acquisition of the lock that
143
146
we now hold.
144
147
145
- o Before mod_timer() time for a timer handler.
148
+ - Before mod_timer() time for a timer handler.
146
149
147
150
There are many other possibilities involving the Linux
148
151
kernel's wide array of primitives that cause code to
149
152
be invoked at a later time.
150
153
151
- o The pointer being compared against also came from
154
+ - The pointer being compared against also came from
152
155
rcu_dereference(). In this case, both pointers depend
153
156
on one rcu_dereference() or another, so you get proper
154
157
ordering either way.
@@ -159,13 +162,13 @@ o Be very careful about comparing pointers obtained from
159
162
of such an RCU usage bug is shown in the section titled
160
163
"EXAMPLE OF AMPLIFIED RCU-USAGE BUG".
161
164
162
- o All of the accesses following the comparison are stores,
165
+ - All of the accesses following the comparison are stores,
163
166
so that a control dependency preserves the needed ordering.
164
167
That said, it is easy to get control dependencies wrong.
165
168
Please see the "CONTROL DEPENDENCIES" section of
166
169
Documentation/memory-barriers.txt for more details.
167
170
168
- o The pointers are not equal -and- the compiler does
171
+ - The pointers are not equal -and- the compiler does
169
172
not have enough information to deduce the value of the
170
173
pointer. Note that the volatile cast in rcu_dereference()
171
174
will normally prevent the compiler from knowing too much.
@@ -175,7 +178,7 @@ o Be very careful about comparing pointers obtained from
175
178
comparison will provide exactly the information that the
176
179
compiler needs to deduce the value of the pointer.
177
180
178
- o Disable any value-speculation optimizations that your compiler
181
+ - Disable any value-speculation optimizations that your compiler
179
182
might provide, especially if you are making use of feedback-based
180
183
optimizations that take data collected from prior runs. Such
181
184
value-speculation optimizations reorder operations by design.
@@ -188,11 +191,12 @@ o Disable any value-speculation optimizations that your compiler
188
191
189
192
190
193
EXAMPLE OF AMPLIFIED RCU-USAGE BUG
194
+ ----------------------------------
191
195
192
196
Because updaters can run concurrently with RCU readers, RCU readers can
193
197
see stale and/or inconsistent values. If RCU readers need fresh or
194
198
consistent values, which they sometimes do, they need to take proper
195
- precautions. To see this, consider the following code fragment:
199
+ precautions. To see this, consider the following code fragment::
196
200
197
201
struct foo {
198
202
int a;
@@ -244,7 +248,7 @@ to some reordering from the compiler and CPUs is beside the point.
244
248
245
249
But suppose that the reader needs a consistent view?
246
250
247
- Then one approach is to use locking, for example, as follows:
251
+ Then one approach is to use locking, for example, as follows::
248
252
249
253
struct foo {
250
254
int a;
@@ -299,6 +303,7 @@ As always, use the right tool for the job!
299
303
300
304
301
305
EXAMPLE WHERE THE COMPILER KNOWS TOO MUCH
306
+ -----------------------------------------
302
307
303
308
If a pointer obtained from rcu_dereference() compares not-equal to some
304
309
other pointer, the compiler normally has no clue what the value of the
@@ -308,7 +313,7 @@ guarantees that RCU depends on. And the volatile cast in rcu_dereference()
308
313
should prevent the compiler from guessing the value.
309
314
310
315
But without rcu_dereference(), the compiler knows more than you might
311
- expect. Consider the following code fragment:
316
+ expect. Consider the following code fragment::
312
317
313
318
struct foo {
314
319
int a;
@@ -354,6 +359,7 @@ dereference the resulting pointer.
354
359
355
360
356
361
WHICH MEMBER OF THE rcu_dereference() FAMILY SHOULD YOU USE?
362
+ ------------------------------------------------------------
357
363
358
364
First, please avoid using rcu_dereference_raw() and also please avoid
359
365
using rcu_dereference_check() and rcu_dereference_protected() with a
@@ -370,22 +376,22 @@ member of the rcu_dereference() to use in various situations:
370
376
371
377
2. If the access might be within an RCU read-side critical section
372
378
on the one hand, or protected by (say) my_lock on the other,
373
- use rcu_dereference_check(), for example:
379
+ use rcu_dereference_check(), for example::
374
380
375
381
p1 = rcu_dereference_check(p->rcu_protected_pointer,
376
382
lockdep_is_held(&my_lock));
377
383
378
384
379
385
3. If the access might be within an RCU read-side critical section
380
386
on the one hand, or protected by either my_lock or your_lock on
381
- the other, again use rcu_dereference_check(), for example:
387
+ the other, again use rcu_dereference_check(), for example::
382
388
383
389
p1 = rcu_dereference_check(p->rcu_protected_pointer,
384
390
lockdep_is_held(&my_lock) ||
385
391
lockdep_is_held(&your_lock));
386
392
387
393
4. If the access is on the update side, so that it is always protected
388
- by my_lock, use rcu_dereference_protected():
394
+ by my_lock, use rcu_dereference_protected()::
389
395
390
396
p1 = rcu_dereference_protected(p->rcu_protected_pointer,
391
397
lockdep_is_held(&my_lock));
@@ -410,18 +416,19 @@ member of the rcu_dereference() to use in various situations:
410
416
411
417
412
418
SPARSE CHECKING OF RCU-PROTECTED POINTERS
419
+ -----------------------------------------
413
420
414
421
The sparse static-analysis tool checks for direct access to RCU-protected
415
422
pointers, which can result in "interesting" bugs due to compiler
416
423
optimizations involving invented loads and perhaps also load tearing.
417
- For example, suppose someone mistakenly does something like this:
424
+ For example, suppose someone mistakenly does something like this::
418
425
419
426
p = q->rcu_protected_pointer;
420
427
do_something_with(p->a);
421
428
do_something_else_with(p->b);
422
429
423
430
If register pressure is high, the compiler might optimize "p" out
424
- of existence, transforming the code to something like this:
431
+ of existence, transforming the code to something like this::
425
432
426
433
do_something_with(q->rcu_protected_pointer->a);
427
434
do_something_else_with(q->rcu_protected_pointer->b);
@@ -435,7 +442,7 @@ Load tearing could of course result in dereferencing a mashup of a pair
435
442
of pointers, which also might fatally disappoint your code.
436
443
437
444
These problems could have been avoided simply by making the code instead
438
- read as follows:
445
+ read as follows::
439
446
440
447
p = rcu_dereference(q->rcu_protected_pointer);
441
448
do_something_with(p->a);
@@ -448,7 +455,7 @@ or as a formal parameter, with "__rcu", which tells sparse to complain if
448
455
this pointer is accessed directly. It will also cause sparse to complain
449
456
if a pointer not marked with "__rcu" is accessed using rcu_dereference()
450
457
and friends. For example, ->rcu_protected_pointer might be declared as
451
- follows:
458
+ follows::
452
459
453
460
struct foo __rcu *rcu_protected_pointer;
454
461
0 commit comments