Skip to content

Commit 0d75e0c

Browse files
ubizjakPeter Zijlstra
authored andcommitted
locking/osq_lock: Use atomic_try_cmpxchg_release() in osq_unlock()
Replace this pattern in osq_unlock(): atomic_cmpxchg(*ptr, old, new) == old ... with the simpler and faster: atomic_try_cmpxchg(*ptr, &old, new) The x86 CMPXCHG instruction returns success in the ZF flag, so this change saves a compare after the CMPXCHG. The code in the fast path of osq_unlock() improves from: 11b: 31 c9 xor %ecx,%ecx 11d: 8d 50 01 lea 0x1(%rax),%edx 120: 89 d0 mov %edx,%eax 122: f0 0f b1 0f lock cmpxchg %ecx,(%rdi) 126: 39 c2 cmp %eax,%edx 128: 75 05 jne 12f <...> to: 12b: 31 d2 xor %edx,%edx 12d: 83 c0 01 add $0x1,%eax 130: f0 0f b1 17 lock cmpxchg %edx,(%rdi) 134: 75 05 jne 13b <...> Signed-off-by: Uros Bizjak <[email protected]> Signed-off-by: Peter Zijlstra (Intel) <[email protected]> Acked-by: Waiman Long <[email protected]> Link: https://lore.kernel.org/r/[email protected]
1 parent f730fd5 commit 0d75e0c

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

kernel/locking/osq_lock.c

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -215,8 +215,7 @@ void osq_unlock(struct optimistic_spin_queue *lock)
215215
/*
216216
* Fast path for the uncontended case.
217217
*/
218-
if (likely(atomic_cmpxchg_release(&lock->tail, curr,
219-
OSQ_UNLOCKED_VAL) == curr))
218+
if (atomic_try_cmpxchg_release(&lock->tail, &curr, OSQ_UNLOCKED_VAL))
220219
return;
221220

222221
/*

0 commit comments

Comments
 (0)