Clarify the difference between Lock and RLock when the acquiring thread returns

[lzlarryli]

Documentation

Because RLock has an "owner", it is fully released when its acquiring thread exits, no matter how many times it is acquired. The ownerless Lock on the other hand, stays locked when its acquiring thread exits. This should be documented because it can lead to hard to debug errors.

Example code to see this difference:

def f(lock):
  print('f lock:', lock.acquire(False))
  print('f relock:', lock.acquire(False))
    

def g(lock):
  print('g lock:', lock.acquire(False))
  print('g relock:', lock.acquire(False))


print('Lock:')
lock = threading.Lock()
t1 = threading.Thread(target=f, args=(lock,))
t2 = threading.Thread(target=g, args=(lock,))
t1.start()
t1.join()
t2.start()
t2.join()
print()

print('RLock:')
lock = threading.RLock()
t1 = threading.Thread(target=f, args=(lock,))
t2 = threading.Thread(target=g, args=(lock,))
t1.start()
t1.join()
t2.start()
t2.join()

Output:

Lock:
f lock: True
f relock: False
g lock: False
g relock: False

RLock:
f lock: True
f relock: True
g lock: True
g relock: True

Linked PRs

• gh-128167: Document another difference between RLock and Lock #128168

[picnixz]

I'm not sure I follow what the issue is. When a thread tries to acquire a lock, it also tries to steal ownership of that lock. In particular, a re-entrant lock is not released, at least not in the pure Python implementation (you can see that the recursion count at the end of the program is 4 for the re-entrant lock if you use threading._CRLock = None before and check lock._count).

I didn't check the C implementation but it's probably the same. For instance:

lock = threading.RLock()
t1 = threading.Thread(target=f, args=(lock,))
t2 = threading.Thread(target=g, args=(lock,))
t1.start()
t1.join()
assert lock.acquire(False) is False  # main thread failed to acquire the lock!

t2.start()
t2.join()

assert lock.acquire(False) is False  # main thread failed to acquire the lock!
lock.release()  # will fail with a RuntimeError even though t2 is done.

EDIT: I'm wondering why the main thread cannot acquire the lock but t2 can actually. Is there some magic I'm not aware of? cc @gpshead as the threading/mp expert.

[lzlarryli]

OK. I dug into the code and found that I was wrong. The problem is caused instead by the immediate recycling of threading.get_ident(). Because t2 is started immediately after t1 is joined, threading.get_ident() in t2 got the same number as that in t1, to the RLock t2 is the same owner. It seems threading.get_ident() can be made more robust but that is a different issue.

class MyRLock:
  """A simplified RLock.
  
  Taken from https://github.com/python/cpython/blob/main/Lib/threading.py#L126
  """
  def __init__(self):
    self.lock = threading.Lock()
    self.owner = None
    self.count = 0
  
  def acquire(self, blocking=True):
    tid = threading.get_ident()
    print('thread ID:', tid, 'Lock owner:', self.owner, flush=True)
    if self.owner == tid:
      self.count += 1
      return True
    acquired = self.lock.acquire(blocking)
    if acquired:
      self.owner = tid
      self.count = 1
      return True
    return False
  
  def release(self):
    tid = threading.get_ident()
    with self.statelock:
      if self.owner != tid:
        raise RuntimeError('Cannot release unacquired lock.')
      self.count -= 1
      if self.count == 0:
        self.owner = None
        self.lock.release()

def f(lock):
  print('f lock:', lock.acquire(False))
  print('f relock:', lock.acquire(False))

def g(lock):
  print('g lock:', lock.acquire(False))
  print('g relock:', lock.acquire(False))

lock = MyRLock()
t1 = threading.Thread(target=f, args=(lock,))
t2 = threading.Thread(target=g, args=(lock,))
t1.start()
t1.join()
print('main lock:', lock.acquire(False))

t2.start()
t2.join()
print('main lock:', lock.acquire(False))

The output is:

thread ID: 132671976138304 Lock owner: None
f lock: True
thread ID: 132671976138304 Lock owner: 132671976138304
f relock: True
thread ID: 132673314148352 Lock owner: 132671976138304
main lock: False
thread ID: 132671976138304 Lock owner: 132671976138304
g lock: True
thread ID: 132671976138304 Lock owner: 132671976138304
g relock: True
thread ID: 132673314148352 Lock owner: 132671976138304
main lock: False

[picnixz]

Considering this is the Python implementation of locks, I don't think we need to make it more robust but I'll leave it at @gpshead's discretion.