CurrentThreadExecutor already quit or is broken

[novucs]

Problem

RuntimeError: CurrentThreadExecutor already quit or is broken is raised on tasks that are spawned from a request and use sync_to_async after the spawning request is complete.

I'm aware background tasks should normally be managed by celery or other. Asyncio tasks were a handy stopgap here as the application I'm working on is only ever used locally on desktop.

Similar issue: #491

Affected Versions

Versions 3.9.0 and 3.9.1 are producing the error.

Everything worked as expected on version 3.8.1

Minimal Example

# app.py
import asyncio
import traceback

from asgiref.sync import sync_to_async
from nanodjango import Django

app = Django()


@app.route("/")
async def index(request):
    async def inner_task():
        await asyncio.sleep(0.1)  # Wait for request to complete
        try:
            await sync_to_async(lambda: ...)()
        except Exception:
            traceback.print_exc()
            raise
        print("Sync function executed successfully!")

    asyncio.create_task(inner_task())
    return "Check console for errors"

# Working as expected
uvx --with 'asgiref==3.8.1' nanodjango run app.py

# Produces errors in console
uvx --with 'asgiref==3.9.0' nanodjango run app.py
uvx --with 'asgiref==3.9.1' nanodjango run app.py

[novucs]

Changing

            await sync_to_async(lambda: ...)()

To:

            await sync_to_async(lambda: ..., thread_sensitive=False)()

Appears to work for 3.9.0 and onwards

[carltongibson]

cc @andersk

[andersk]

git bisect shows that this case stopped working with

• Fix Local behaviour with asyncio Task.  #478

Before that, current_thread_executor was None here in SyncToAsync.__call__:

asgiref/asgiref/sync.py

Lines 419 to 422 in 8e39bcc

	current_thread_executor = getattr(AsyncToSync.executors, "current", None)
	if current_thread_executor:
	# If we have a parent sync thread above somewhere, use that
	executor = current_thread_executor

and so we created a ThreadPoolExecutor instead of a CurrentThreadExecutor:

asgiref/asgiref/sync.py

Lines 432 to 433 in 8e39bcc

	# Create new thread executor in current context
	executor = ThreadPoolExecutor(max_workers=1)

I think the new behavior is correct. Either way, we’re unable to put you back into the same sync thread that has already exited, but now at least we inform you of this instead of silently running your code in the wrong thread. If you know that your code is thread-safe, thread_sensitive=False is the correct way to request the old behavior.

[andersk]

By the way, you probably want to heed this warning in the documentation for asyncio.create_task:

Important: Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done.

[novucs]

Thanks for the explainer, good to know this was never intended functionality to start with!

By the way, you probably want to heed this warning in the documentation for asyncio.create_task

Thanks for this, I omitted the task reference in my minimal example to make it easier to comprehend & it appeared to still produce the same error.

I'll close this now as my question is answered. Good to know this antipattern was even more flawed than I'd initially anticipated! 😂

[iwakitakuma33]

Since I investigated the same phenomenon, I will leave the investigation results here in case others encounter the same phenomenon.

When creating tasks with queuing servers such as Celery, there is no problem because Celery continues to run.
However, when creating tasks asynchronously with Django requests, caution is required.
Until 3.8.1, even if the Django request ended, the created task would not end, but in 3.9.0 and later, if the parent thread ends, the task will also be treated as ended.
The behavior in 3.9.0 and later is preferable. The reason is to maintain an independent lifecycle for each request.
Setting up a queuing server that executes asynchronous tasks independent of requests is safer in terms of resource usage.

[carltongibson]

@andersk do you think a tweak to the changelog would help here? 🤔