Skip to content

keyring >=24.0.0 on macos somehow corrupts multiprocessing #713

@Roguelazer

Description

@Roguelazer

Describe the bug
This is one of the weirder bugs that I've ever run into. If you use the keyring module to read a credential on macOS and then subsequently close a multiprocessing.ThreadPool instance during interpreter shutdown, the multiprocessing module crashes.

To Reproduce

Run the following script:

import keyring
# this is load-bearing: if you don't import multiprocessing first, everything works. some kind
# of reference-counting issue?
import multiprocessing
from multiprocessing.pool import ThreadPool

class Repro:
    def __init__(self):
        self.pool = ThreadPool()

    def __del__(self):
        if self.pool is not None:
            self.pool.close()
            self.pool.join()
            self.pool = None


def main():
    # comment out the call to keyring, and everything's fine
    keyring.get_password("example", "example-credential")
    # this needs to be bound to a variable so that __del__ runs at shutdown; if you just do
    # Repro(token), the destructor runs immediately and it doesn't exhibit the bug
    client = Repro()


main()

Expected behavior
The program should not print a traceback or crash

Actual behavior
The program reliably prints a warning like

Exception ignored in: <function Repro.__del__ at 0x1059fa520>
Traceback (most recent call last):
  File "./test_mp.py", line 13, in __del__
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 652, in close
  File "/opt/homebrew/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/queues.py", line 371, in put
AttributeError: 'NoneType' object has no attribute 'dumps'

Environment

  • OS: macOS
  • Python Version: Tested 3.11, 3.12, and 31.3
./venv311/bin/pip list | grep keyring
keyring            25.6.0
$ ./venv311/bin/keyring --list-backends
keyring.backends.fail.Keyring (priority: 0)
keyring.backends.macOS.Keyring (priority: 5)
keyring.backends.chainer.ChainerBackend (priority: -1)

Additional context
I tested with various keyring versions and this was introduced in 24.0.0; version 23.13.1 does not exhibit the problem.

I read the diff between them and... nothing jumps out to me...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions