Skip to content

Conversation

andersk
Copy link
Contributor

@andersk andersk commented Feb 13, 2025

A CurrentThreadExecutor may terminate with work still remaining in its queue, or new work may be submitted later. We previously discarded remaining work, leading to deadlocks, and raised an error on submitting late work. But if there’s another CurrentThreadExecutor for the same thread below us on the stack, we should instead migrate the extra work there to allow it to eventually run.

Doing this in a thread-safe way requires replacing the queue.Queue abstraction with collections.deque and threading.ConditionVariable (the same primitives used to implement queue.Queue).

@andersk andersk force-pushed the CurrentThreadExecutor-migrate branch from 8a2717c to 2a7ba5e Compare June 29, 2025 14:49
A CurrentThreadExecutor may terminate with work still remaining in its
queue, or new work may be submitted later.  We previously discarded
remaining work, leading to deadlocks, and raised an error on
submitting late work.  Instead, execute remaining work immediately,
and if there’s another CurrentThreadExecutor for the same thread below
us on the stack, redirect late work there to allow it to eventually
run.

Doing this in a thread-safe way requires replacing the queue.Queue
abstraction with collections.deque and
threading.ConditionVariable (the same primitives used to implement
queue.Queue).

Fixes django#491; fixes django#492.

Signed-off-by: Anders Kaseorg <[email protected]>
@andersk andersk force-pushed the CurrentThreadExecutor-migrate branch from 2a7ba5e to 3ece1c9 Compare June 30, 2025 19:34


def test_two_nested_tasks_with_asyncio_run() -> None:
barrier = asyncio.Barrier(3)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to skip this test on <PY311. I think that's OK.

@bellini666
Copy link
Contributor

I'm really looking forward to see this merged and a new release get out so I can fix the translations issues at my work.

Let me know if any of you need any help in making that happening :)

e.g. I could add the missing skip to the PR, but I don't have permissions for that 😅

Copy link
Member

@carltongibson carltongibson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work here @andersk — looks good.

I just pushed some edits to get the tests/mypy passing for Python <3.11. Can you confirm if you're happy with those?

👍

@andersk
Copy link
Contributor Author

andersk commented Jul 2, 2025

Yeah that looks good to me. Thanks!

I can look into fixing up the test for older Python later, but we don’t need to block on that.

@carltongibson
Copy link
Member

@andersk Super. Thanks for confirming. I'll get this in tomorrow now and look for a release by the end of the week.

Thanks for the effort, and to @bellini666 too for your review (which got me over the dread-hump 🥳)

@carltongibson carltongibson merged commit ef1678a into django:main Jul 3, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants