asgiref 3.9.0 breaks django-hosts

[phihag]

asgiref 3.9.0 in conjunction with django-hosts 7.0.0 (their latest version) makes between 10% and 20% of requests to our admin host fail with

NoReverseMatch: 'admin' is not a registered namespace

These requests are calling something like reverse('admin:login'), but the django-hosts configuration is the fallback one for their thread.

Reverting to asgiref 3.8.1 fixes the problem.

I suspect this code change, which uses a normal dictionary for thread-local access: 85d2445#r161477713

[carltongibson]

@phihag Can you reduce this to some kind of reproduction? (A failing test case would be perfect)

[phihag]

I will try. It's a bit tricky since it only occurs if there is other traffic. The actual failures are on every page that calls reverse, including e.g. the login page.

The change I linked is definitely wrong though, right? It sets in a normal dict from multiple threads, in a construct that's explicitly designed to use thread-local storage.

[carltongibson]

OK, I'll look at it this week. 👀 //cc @bellini666 @spanezz

[bellini666]

I can see why this can happen. The current solution allows the ContextVar to be deleted, meaning one thread/ctx can remove the context from all others, which is probably what is happening here, as django-hosts seems to be triggering this

I just opened this PR which reviews the solution with a .copy(), as discussed in here, and added a regression test for this case, while ensuring the existing tests that got added during that PR and related one still passes

[sttwister]

A silent upgrade to 3.9.0 also impacted us in production with similar results - a percentage of requests threw errors resolving Django URLs.

In our case, a middleware was setting request.urlconf to a different value, depending on the request host. Concurrent requests somehow invalidated setting this custom urlconf. I assume this is similar to what django-hosts does.

[carltongibson]

@phihag @sttwister Can you try the branch from #523 please, to see if it resolved your issue?

[carltongibson]

It looks like the diagnosis is correct. Django resets the urlconf after each request...

def reset_urlconf(sender, **kwargs):
    """Reset the URLconf after each request is finished."""
    set_urlconf(None)


request_finished.connect(reset_urlconf)

... which would trigger the case here.

[bellini666]

@carltongibson I was curious on why this hasn't affected us at all. We've been running 3.9.0 since its release, in an app that gets ~3 million requests/day, and no issues at all.

After checking django's code in detail, this only affects situations where urlconf is modified in the request by setting request.urlconf, which is then set in the current thread/context by this condition, and used by the handler to resolve the view and/or a reverse call inside the view itself.

For anyone not setting requests.urlconf, it doesn't cause any issues, as the get_resolver will fallback to the base one, only downside being that it might have lost its cache in the process.

So, at least for urlconf, this requires a very specific situation, like what django-hosts does, as the default implementation is not affected. Changing the solution to .copy() should fix the issue, as setting any value in there would create a copy of the dict during that request, meaning the del _urlconfs.value would only delete the key from that dict, not affecting other requests

[carltongibson]

Hey @bellini666 — yes, exactly. I think we're in agreement.

I would still prefer if either @phihag or @sttwister could try the PR branch to confirm the fix… — is that possible folks? (A "No" is more helpful than silence. 🙏)

[phihag]

I've been looking into it, but it's not really feasible for us to deploy a custom asgiref, sorry. We don't even use it directly, it's just a transitive dependency of ours.

Of course, we could in principle do it, but it's a lot of engineering effort that we cannot spend in the short term. Apologies.

[carltongibson]

@phihag OK, thanks.

For reference the requirements line needed would be:

asgiref @ git+https://github.com/django/asgiref.git@refs/pull/523/head

and the pip command:

pip install 'asgiref @ git+https://github.com/django/asgiref.git@refs/pull/523/head'

VCS Support - Pip Documentation v25.1.1

[carltongibson]

OK, please upgrade to 3.9.1

https://pypi.org/project/asgiref/3.9.1/