Skip to content

Commit e5e726f

Browse files
committed
Merge tag 'locking-core-2021-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking and atomics updates from Thomas Gleixner: "The regular pile: - A few improvements to the mutex code - Documentation updates for atomics to clarify the difference between cmpxchg() and try_cmpxchg() and to explain the forward progress expectations. - Simplification of the atomics fallback generator - The addition of arch_atomic_long*() variants and generic arch_*() bitops based on them. - Add the missing might_sleep() invocations to the down*() operations of semaphores. The PREEMPT_RT locking core: - Scheduler updates to support the state preserving mechanism for 'sleeping' spin- and rwlocks on RT. This mechanism is carefully preserving the state of the task when blocking on a 'sleeping' spin- or rwlock and takes regular wake-ups targeted at the same task into account. The preserved or updated (via a regular wakeup) state is restored when the lock has been acquired. - Restructuring of the rtmutex code so it can be utilized and extended for the RT specific lock variants. - Restructuring of the ww_mutex code to allow sharing of the ww_mutex specific functionality for rtmutex based ww_mutexes. - Header file disentangling to allow substitution of the regular lock implementations with the PREEMPT_RT variants without creating an unmaintainable #ifdef mess. - Shared base code for the PREEMPT_RT specific rw_semaphore and rwlock implementations. Contrary to the regular rw_semaphores and rwlocks the PREEMPT_RT implementation is writer unfair because it is infeasible to do priority inheritance on multiple readers. Experience over the years has shown that real-time workloads are not the typical workloads which are sensitive to writer starvation. The alternative solution would be to allow only a single reader which has been tried and discarded as it is a major bottleneck especially for mmap_sem. Aside of that many of the writer starvation critical usage sites have been converted to a writer side mutex/spinlock and RCU read side protections in the past decade so that the issue is less prominent than it used to be. - The actual rtmutex based lock substitutions for PREEMPT_RT enabled kernels which affect mutex, ww_mutex, rw_semaphore, spinlock_t and rwlock_t. The spin/rw_lock*() functions disable migration across the critical section to preserve the existing semantics vs per-CPU variables. - Rework of the futex REQUEUE_PI mechanism to handle the case of early wake-ups which interleave with a re-queue operation to prevent the situation that a task would be blocked on both the rtmutex associated to the outer futex and the rtmutex based hash bucket spinlock. While this situation cannot happen on !RT enabled kernels the changes make the underlying concurrency problems easier to understand in general. As a result the difference between !RT and RT kernels is reduced to the handling of waiting for the critical section. !RT kernels simply spin-wait as before and RT kernels utilize rcu_wait(). - The substitution of local_lock for PREEMPT_RT with a spinlock which protects the critical section while staying preemptible. The CPU locality is established by disabling migration. The underlying concepts of this code have been in use in PREEMPT_RT for way more than a decade. The code has been refactored several times over the years and this final incarnation has been optimized once again to be as non-intrusive as possible, i.e. the RT specific parts are mostly isolated. It has been extensively tested in the 5.14-rt patch series and it has been verified that !RT kernels are not affected by these changes" * tag 'locking-core-2021-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (92 commits) locking/rtmutex: Return success on deadlock for ww_mutex waiters locking/rtmutex: Prevent spurious EDEADLK return caused by ww_mutexes locking/rtmutex: Dequeue waiter on ww_mutex deadlock locking/rtmutex: Dont dereference waiter lockless locking/semaphore: Add might_sleep() to down_*() family locking/ww_mutex: Initialize waiter.ww_ctx properly static_call: Update API documentation locking/local_lock: Add PREEMPT_RT support locking/spinlock/rt: Prepare for RT local_lock locking/rtmutex: Add adaptive spinwait mechanism locking/rtmutex: Implement equal priority lock stealing preempt: Adjust PREEMPT_LOCK_OFFSET for RT locking/rtmutex: Prevent lockdep false positive with PI futexes futex: Prevent requeue_pi() lock nesting issue on RT futex: Simplify handle_early_requeue_pi_wakeup() futex: Reorder sanity checks in futex_requeue() futex: Clarify comment in futex_requeue() futex: Restructure futex_requeue() futex: Correct the number of requeued waiters for PI futex: Remove bogus condition for requeue PI ...
2 parents 08403e2 + a055fcc commit e5e726f

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

76 files changed

+5941
-2791
lines changed

Documentation/atomic_t.txt

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,3 +271,97 @@ WRITE_ONCE. Thus:
271271
SC *y, t;
272272

273273
is allowed.
274+
275+
276+
CMPXCHG vs TRY_CMPXCHG
277+
----------------------
278+
279+
int atomic_cmpxchg(atomic_t *ptr, int old, int new);
280+
bool atomic_try_cmpxchg(atomic_t *ptr, int *oldp, int new);
281+
282+
Both provide the same functionality, but try_cmpxchg() can lead to more
283+
compact code. The functions relate like:
284+
285+
bool atomic_try_cmpxchg(atomic_t *ptr, int *oldp, int new)
286+
{
287+
int ret, old = *oldp;
288+
ret = atomic_cmpxchg(ptr, old, new);
289+
if (ret != old)
290+
*oldp = ret;
291+
return ret == old;
292+
}
293+
294+
and:
295+
296+
int atomic_cmpxchg(atomic_t *ptr, int old, int new)
297+
{
298+
(void)atomic_try_cmpxchg(ptr, &old, new);
299+
return old;
300+
}
301+
302+
Usage:
303+
304+
old = atomic_read(&v); old = atomic_read(&v);
305+
for (;;) { do {
306+
new = func(old); new = func(old);
307+
tmp = atomic_cmpxchg(&v, old, new); } while (!atomic_try_cmpxchg(&v, &old, new));
308+
if (tmp == old)
309+
break;
310+
old = tmp;
311+
}
312+
313+
NB. try_cmpxchg() also generates better code on some platforms (notably x86)
314+
where the function more closely matches the hardware instruction.
315+
316+
317+
FORWARD PROGRESS
318+
----------------
319+
320+
In general strong forward progress is expected of all unconditional atomic
321+
operations -- those in the Arithmetic and Bitwise classes and xchg(). However
322+
a fair amount of code also requires forward progress from the conditional
323+
atomic operations.
324+
325+
Specifically 'simple' cmpxchg() loops are expected to not starve one another
326+
indefinitely. However, this is not evident on LL/SC architectures, because
327+
while an LL/SC architecure 'can/should/must' provide forward progress
328+
guarantees between competing LL/SC sections, such a guarantee does not
329+
transfer to cmpxchg() implemented using LL/SC. Consider:
330+
331+
old = atomic_read(&v);
332+
do {
333+
new = func(old);
334+
} while (!atomic_try_cmpxchg(&v, &old, new));
335+
336+
which on LL/SC becomes something like:
337+
338+
old = atomic_read(&v);
339+
do {
340+
new = func(old);
341+
} while (!({
342+
volatile asm ("1: LL %[oldval], %[v]\n"
343+
" CMP %[oldval], %[old]\n"
344+
" BNE 2f\n"
345+
" SC %[new], %[v]\n"
346+
" BNE 1b\n"
347+
"2:\n"
348+
: [oldval] "=&r" (oldval), [v] "m" (v)
349+
: [old] "r" (old), [new] "r" (new)
350+
: "memory");
351+
success = (oldval == old);
352+
if (!success)
353+
old = oldval;
354+
success; }));
355+
356+
However, even the forward branch from the failed compare can cause the LL/SC
357+
to fail on some architectures, let alone whatever the compiler makes of the C
358+
loop body. As a result there is no guarantee what so ever the cacheline
359+
containing @v will stay on the local CPU and progress is made.
360+
361+
Even native CAS architectures can fail to provide forward progress for their
362+
primitive (See Sparc64 for an example).
363+
364+
Such implementations are strongly encouraged to add exponential backoff loops
365+
to a failed CAS in order to ensure some progress. Affected architectures are
366+
also strongly encouraged to inspect/audit the atomic fallbacks, refcount_t and
367+
their locking primitives.

drivers/staging/media/atomisp/pci/atomisp_ioctl.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1904,8 +1904,8 @@ int __atomisp_streamoff(struct file *file, void *fh, enum v4l2_buf_type type)
19041904
dev_dbg(isp->dev, "Stop stream on pad %d for asd%d\n",
19051905
atomisp_subdev_source_pad(vdev), asd->index);
19061906

1907-
BUG_ON(!rt_mutex_is_locked(&isp->mutex));
1908-
BUG_ON(!mutex_is_locked(&isp->streamoff_mutex));
1907+
lockdep_assert_held(&isp->mutex);
1908+
lockdep_assert_held(&isp->streamoff_mutex);
19091909

19101910
if (type != V4L2_BUF_TYPE_VIDEO_CAPTURE) {
19111911
dev_dbg(isp->dev, "unsupported v4l2 buf type\n");

0 commit comments

Comments
 (0)