Documentation doesn't generate with docutils >= 0.22

[danigm]

Documentation

The html doc is not building with the new docutils >= 0.22. It's not a problem yet, because the current Sphinx used (sphinx~=8.2.0) doesn't install the latest one, but it's something that should be fixed for the future.

The problem can be reproduced:

$ cd Doc
$ python3 -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt
$ pip install -U docutils
$ sphinx-build --pdb -b html . build/html
[...]
      File "env/lib64/python3.13/site-packages/docutils/parsers/rst/states.py", line 1494, in parse_enumerator
        ordinal = int(self.enum.converters[sequence](text))
    TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

The actual problem is directly related to this line in docutils, done as part as this commit.

They wrap the converters with int so the monkey patch done in pyspecific.py doesn't work anymore.

# monkey-patch reST parser to disable alphabetic and roman enumerated lists
from docutils.parsers.rst.states import Body
Body.enum.converters['loweralpha'] = \
    Body.enum.converters['upperalpha'] = \
    Body.enum.converters['lowerroman'] = \
    Body.enum.converters['upperroman'] = lambda x: None

Linked PRs

• gh-139257: Support docutils >= 0.22 #139258

[picnixz]

I don't think we should actually worry about a docutils version that is not installed by the environment itself. I would rather pin the docutils version in constraints.txt instead of re-patching conf.py directly. It might not be the only issue we'll have if we upgrade to newest Sphinx.

Is there a reason why the docutils version is actually being updated after installing the requirements? maybe we should pin the docutils version anyway so that it's compatible with our conf.py (having version-guarded ifs always complicates maintenance).

[danigm]

I don't think we should actually worry about a docutils version that is not installed by the environment itself. I would rather pin the docutils version in constraints.txt instead of re-patching conf.py directly. It might not be the only issue we'll have if we upgrade to newest Sphinx.

Is there a reason why the docutils version is actually being updated after installing the requirements? maybe we should pin the docutils version anyway so that it's compatible with our conf.py (having version-guarded ifs always complicates maintenance).

This problem appears in openSUSE where we are trying to update docutils, and we can patch downstream to keep it working. But as I said, this is not a problem right now, but will be a problem in the future, so I tried to work on a fix for it.

[picnixz]

Ah I see in the logs

[ 7s] [169/192] cumulate python313-docutils-0.22.2-1.2

So the venv you're using for building the docs on openSUSE is actually using the latest docutils and the latest Sphinx (for some reasons) instead of those given in Doc/requirements.txt. How easy is it to actually use those instead? I would prefer pinning the versions in Doc/requirements.txt and Doc/constraints.txt instead as other users might benefit from this locally. If this is not easy, I can indeed accept the docutils patch (other distros might have the same issue)

[danigm]

That is the main issue. We build cpython in openSUSE with the current packages, everything installed from the rpm repository so there's no venv during build. That's why if we want to update docutils we need to patch cpython to work correctly with the new version.

For documentation usually the Sphinx version is the problem and we are more conservative when updating Sphinx, but in this case, it's docutils and this specific monkey-patch that doesn't work with the current version.

The patch is not very complex and it's compatible with the previous versions, so I though that it could be interesting to report and fix upstream.

[picnixz]

Mmh. Are you aware of other distributors that could have a similar issue? I don't know much about their process. If all other major distributors also don't use our venv, then I think we could allow this patch (or we can also change upstream docutils, cc @AA-Turner).

Considering the issue, I think I'd be fine with the version-guarded patch.

[AA-Turner]

We don't support building the documentation with exotic configurations/environments. When we update to a newer version of Sphinx, we'll address this. Thank you for raising the issue, though.

A

[mcepl]

Mmh. Are you aware of other distributors that could have a similar issue? I don't know much about their process. If all other major distributors also don't use our venv, then I think we could allow this patch (or we can also change upstream docutils, cc @AA-Turner).

I think this will be the problem for all Linux distributions (and others like *BSD, Homebrew on Mac, etc.), only some of them have so far old docutils (e.g., Fedora is still on 0.21.2, Debian will hits this in experiemental).

[picnixz]

@AA-Turner I think we could make the change here otherwise it could complicate redistribution. We're already somehow monkey-patching docutils so maybe it doesn't hurt to monkey-patch it better (the fix looks fine, and instead of an ImportError, we could use docutils' version guards instead).