Skip to content

Commit 0aac9da

Browse files
Waiman-LongNeeraj Upadhyay
authored andcommitted
rcu: Use system_unbound_wq to avoid disturbing isolated CPUs
It was discovered that isolated CPUs could sometimes be disturbed by kworkers processing kfree_rcu() works causing higher than expected latency. It is because the RCU core uses "system_wq" which doesn't have the WQ_UNBOUND flag to handle all its work items. Fix this violation of latency limits by using "system_unbound_wq" in the RCU core instead. This will ensure that those work items will not be run on CPUs marked as isolated. Beside the WQ_UNBOUND flag, the other major difference between system_wq and system_unbound_wq is their max_active count. The system_unbound_wq has a max_active of WQ_MAX_ACTIVE (512) while system_wq's max_active is WQ_DFL_ACTIVE (256) which is half of WQ_MAX_ACTIVE. Reported-by: Vratislav Bendel <[email protected]> Closes: https://issues.redhat.com/browse/RHEL-50220 Signed-off-by: Waiman Long <[email protected]> Reviewed-by: "Uladzislau Rezki (Sony)" <[email protected]> Tested-by: Breno Leitao <[email protected]> Signed-off-by: Neeraj Upadhyay <[email protected]>
1 parent 8400291 commit 0aac9da

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

kernel/rcu/tree.c

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3539,10 +3539,10 @@ schedule_delayed_monitor_work(struct kfree_rcu_cpu *krcp)
35393539
if (delayed_work_pending(&krcp->monitor_work)) {
35403540
delay_left = krcp->monitor_work.timer.expires - jiffies;
35413541
if (delay < delay_left)
3542-
mod_delayed_work(system_wq, &krcp->monitor_work, delay);
3542+
mod_delayed_work(system_unbound_wq, &krcp->monitor_work, delay);
35433543
return;
35443544
}
3545-
queue_delayed_work(system_wq, &krcp->monitor_work, delay);
3545+
queue_delayed_work(system_unbound_wq, &krcp->monitor_work, delay);
35463546
}
35473547

35483548
static void
@@ -3634,7 +3634,7 @@ static void kfree_rcu_monitor(struct work_struct *work)
36343634
// be that the work is in the pending state when
36353635
// channels have been detached following by each
36363636
// other.
3637-
queue_rcu_work(system_wq, &krwp->rcu_work);
3637+
queue_rcu_work(system_unbound_wq, &krwp->rcu_work);
36383638
}
36393639
}
36403640

@@ -3704,7 +3704,7 @@ run_page_cache_worker(struct kfree_rcu_cpu *krcp)
37043704
if (rcu_scheduler_active == RCU_SCHEDULER_RUNNING &&
37053705
!atomic_xchg(&krcp->work_in_progress, 1)) {
37063706
if (atomic_read(&krcp->backoff_page_cache_fill)) {
3707-
queue_delayed_work(system_wq,
3707+
queue_delayed_work(system_unbound_wq,
37083708
&krcp->page_cache_work,
37093709
msecs_to_jiffies(rcu_delay_page_cache_fill_msec));
37103710
} else {

0 commit comments

Comments
 (0)