django-rest-framework modifies global docutils rst parser

[nijel]

django-rest-framework relies on django.contrib.admindocs on some regex cleanups:

django-rest-framework/rest_framework/schemas/generators.py

Line 10 in b483179

from django.contrib.admindocs.views import simplify_regex

Unfortunately, importing django.contrib.admindocs.views imports django.contrib.admindocs.utils which has a side effect that it customizes the rst parser in docutils:

https://github.com/django/django/blob/c28f821c9067050ba0d099349a4dfea2b29faf99/django/contrib/admindocs/utils.py#L164-L170

The root cause of this being that docutils does only have a global registry (https://sourceforge.net/p/docutils/feature-requests/38/ is open for more than 10 years).

Would it be possible to use a different way to simplify regular expressions that doesn't have such side effects?

Checklist

• Raised initially as discussion #...
• This is not a feature request suitable for implementation outside this project. Please elaborate what it is:
  • compatibility fix for new Django/Python version ...
  • other type of bug fix
  • other type of improvement that does not touch existing code or change existing behavior (e.g. wrapper for new Django field)
• I have reduced the issue to the simplest possible case.

[RehanK-123]

I would like to solve this issue, if you would allow me to

[browniebroke]

it sounds to me that the problem stems from the fact that "django.contrib.admindocs.utils[...] has a side effect that it customizes the rst parser in docutils" and we use other stuff unrelated to docutils in this module.

Have you tried to raise this problem upstream to the Django project itself? I'm guessing that it may be considered an internal API from Django, however by searching GitHub, it's used in multiple other projects. Fixing it upstream would solve the problem not only for DRF users, but also for all users of these others projects.

[nijel]

I've considered that, but I don't see a way for django.contrib.admindocs to work without customizing docutils, while not requiring django.contrib.admindocs in the django-rest-framework seems doable.

On the other side, I didn't investigate whether this usage is wide-spread.

I've created https://code.djangoproject.com/ticket/36124 to get another opinion there.

[browniebroke]

One could move these regex functions outside of the admindocs utils.py file. Django admin docs and DRF would import it from these without the side effect.

Thanks for opening the ticket there!

[browniebroke]

Did you end up creating a forum topic? I opened a PoC PR to see if Django's test suite suite passes: django/django#19116 and if we can run our own test suite against it: #9636

You could try to run some tests against these versions in your own project. I would recommend forking the version of Django you're using and applying the same patch (unless you're installing Django from source 😄)

[nijel]

I finally got to it now: https://forum.djangoproject.com/t/admindocs-modifies-docutils-behavior-on-import/38637