Skip to content

Commit 33c0860

Browse files
Frederic Weisbeckerpaulmckrcu
authored andcommitted
rcu/exp: Remove superfluous full memory barrier upon first EQS snapshot
When the grace period kthread checks the extended quiescent state counter of a CPU, full ordering is necessary to ensure that either: * If the GP kthread observes the remote target in an extended quiescent state, then that target must observe all accesses prior to the current grace period, including the current grace period sequence number, once it exits that extended quiescent state. or: * If the GP kthread observes the remote target NOT in an extended quiescent state, then the target further entering in an extended quiescent state must observe all accesses prior to the current grace period, including the current grace period sequence number, once it enters that extended quiescent state. This ordering is enforced through a full memory barrier placed right before taking the first EQS snapshot. However this is superfluous because the snapshot is taken while holding the target's rnp lock which provides the necessary ordering through its chain of smp_mb__after_unlock_lock(). Remove the needless explicit barrier before the snapshot and put a comment about the implicit barrier newly relied upon here. Signed-off-by: Frederic Weisbecker <[email protected]> Signed-off-by: Paul E. McKenney <[email protected]> Reviewed-by: Boqun Feng <[email protected]> Reviewed-by: Neeraj Upadhyay <[email protected]> Signed-off-by: Paul E. McKenney <[email protected]>
1 parent 9a7e73c commit 33c0860

File tree

1 file changed

+15
-1
lines changed

1 file changed

+15
-1
lines changed

kernel/rcu/tree_exp.h

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -357,7 +357,21 @@ static void __sync_rcu_exp_select_node_cpus(struct rcu_exp_work *rewp)
357357
!(rnp->qsmaskinitnext & mask)) {
358358
mask_ofl_test |= mask;
359359
} else {
360-
snap = rcu_dynticks_snap(cpu);
360+
/*
361+
* Full ordering between remote CPU's post idle accesses
362+
* and updater's accesses prior to current GP (and also
363+
* the started GP sequence number) is enforced by
364+
* rcu_seq_start() implicit barrier, relayed by kworkers
365+
* locking and even further by smp_mb__after_unlock_lock()
366+
* barriers chained all the way throughout the rnp locking
367+
* tree since sync_exp_reset_tree() and up to the current
368+
* leaf rnp locking.
369+
*
370+
* Ordering between remote CPU's pre idle accesses and
371+
* post grace period updater's accesses is enforced by the
372+
* below acquire semantic.
373+
*/
374+
snap = ct_dynticks_cpu_acquire(cpu);
361375
if (rcu_dynticks_in_eqs(snap))
362376
mask_ofl_test |= mask;
363377
else

0 commit comments

Comments
 (0)