Admonitions in Sphinx 4.4.0 lack closing div

[stevepiercy]

Describe the bug

Admonitions in Sphinx 4.4.0 with MyST lack a closing div. As a result, all content after the admonition becomes part of the admonition content.

Downgrading Sphinx to 4.3.2 resolves the issue.

How to Reproduce

```{note}
This is my note
```

This content gets pulled into the note.

Expected behavior

No response

Your project

https://github.com/plone/documentation

Screenshots

No response

OS

macOS 10.15

Python version

3.10.1

Sphinx version

4.4.0

Sphinx extensions

No response

Extra tools

No response

Additional context

No response

[astrojuanlu]

@stevepiercy does it happen also with reST admonitions?

[stevepiercy]

I tried another project without MyST, upgraded to Sphinx 4.4.0, and the issue does not occur. Also used docutils 0.18.1 for all scenarios.

It must be within the MyST extension or the sphinx-book-theme, in combination with Sphinx 4.4.0. I'm not sure how to isolate it.

Would it help if I were to produce a repo with two branches, one that shows the issue and one that has expected rendering, with a pip requirements file for installation?

[astrojuanlu]

It must be within the MyST extension or the sphinx-book-theme, in combination with Sphinx 4.4.0. I'm not sure how to isolate it.

cc @choldgraf @chrisjsewell

Would it help if I were to produce a repo with two branches, one that shows the issue and one that has expected rendering, with a pip requirements file for installation?

Yes!

[astrojuanlu]

Clarifying whether the issue can be reproduced with the default Sphinx theme (alabaster) would also be useful.

[chrisjsewell]

Also used docutils 0.18.1 for all scenarios.

sphinx is specifically pinned to <0.18, so that could be an issue:

sphinx/setup.py

Line 25 in 2be0630

'docutils>=0.14,<0.18',

I'd have no initial idea why this would be the case,
but it sounds like potentially, like what the expected AST is:

$ myst-docutils-pseudoxml test.md
<document source="test.md">
    <note>
        <paragraph>
            This is my note
    <paragraph>
        This content gets pulled into the note.

and what docutils v0.18 may (?) be producing:

$ myst-docutils-pseudoxml test.md
<document source="test.md">
    <note>
        <paragraph>
            This is my note
        <paragraph>
            This content gets pulled into the note.

[stevepiercy]

Also used docutils 0.18.1 for all scenarios.

sphinx is specifically pinned to <0.18, so that could be an issue:

Yup, that was it.

PyCharm allows me to bypass pip's dependency resolver and install conflicting versions of packages.

But when I install the packages without such foolery, for example pip install -r requirements.txt, then the resolver prevents said foolery.

ERROR: Cannot install docutils==0.18.1 and sphinx==4.4.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested docutils==0.18.1
    sphinx 4.4.0 depends on docutils<0.18 and >=0.14

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
make: *** [build] Error 1

Wow. That is a serious footgun in PyCharm. Reported in:

https://youtrack.jetbrains.com/issue/PY-52623

In any case, this might be something to look out for in the future when you start using a later version of docutils. Here's a screenshot to illustrate the issue.

And here's a repo where I was able to demonstrate and troubleshoot.

https://github.com/stevepiercy/sphinx-myst-admonition

See the README.md for installation instructions for how to replicate.

Feel free to close this issue or add it to the future todos.

[chrisjsewell]

this might be something to look out for in the future when you start using a later version of docutils.

Yeh no worries, thanks for the heads-up
Note, I could not replicate the issue with simply pip install myst-docutils "docutils~=0.18.0"; myst-docutils-pseudoxml test.md
But I will certainly keep an eye out for it

[mwaskom]

I have also encountered this issue, on a site I just created. I did initially start to make the site with pydata_sphinx_theme, but I have rolled back to a very barebones conf.py with the default theme and it persisted (after doing make clean).

I am building the site in a conda virtual environment; interestingly, I have also ended up with the following packages, despite not using PyCharm:

docutils                  0.18.1           py39h6e9494a_0    conda-forge
pydata-sphinx-theme       0.7.2              pyhd8ed1ab_0    conda-forge
sphinx                    3.5.3              pyhd3eb1b0_0
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_1    conda-forge

I installed the packages using mamba, with this command line:

mamba create -n test-pydata-theme python=3.9 pip numpy scipy pandas matplotlib statsmodels pydata-sphinx-theme nbconvert -c conda-forge

I do not know if this is relevant (or an issue on the sphinx / docutils side), but I noticed that at the point where the admonition div should be closed, there is an </aside> tag that has no corresponding opener:

<div class="admonition note">
<p class="admonition-title">Note</p>
<p> ... content ... </p>
</aside>

I hope that might be helpful!

[astrojuanlu]

I see Sphinx 3.5.3 and docutils 0.18.1. looks like the docutils pin was not in place in the conda-forge recipe until Sphinx 4.0:

conda-forge/sphinx-feedstock@73feac1

I would use a lower bound on the Sphinx version to avoid these problems, unless the docutils cap can be retrofitted to all conda-forge versions of Sphinx.

[mwaskom]

Oh interesting, but I wonder why I didn't get sphinx 4+? I created this environment just last weekend.

[mwaskom]

I guess this is probably not a sphinx problem per se but fyi

mamba create -n testsphinx python=3.9 pydata-sphinx-theme -c conda-forge

wants to install these versions (note sphinx < 4):

  Package                            Version  Build               Channel                  Size
─────────────────────────────────────────────────────────────────────────────────────────────────
  Install:
─────────────────────────────────────────────────────────────────────────────────────────────────

  + alabaster                         0.7.12  py_0                conda-forge/noarch     Cached
  + babel                              2.9.1  pyh44b312d_0        conda-forge/noarch     Cached
  + beautifulsoup4                    4.10.0  pyha770c72_0        conda-forge/noarch     Cached
  + brotlipy                           0.7.0  py39h89e85a6_1003   conda-forge/osx-64     Cached
  + bzip2                              1.0.8  h0d85af4_4          conda-forge/osx-64     Cached
  + ca-certificates                2021.10.8  h033912b_0          conda-forge/osx-64     Cached
  + certifi                        2021.10.8  py39h6e9494a_1      conda-forge/osx-64     Cached
  + cffi                              1.15.0  py39he338e87_0      conda-forge/osx-64     Cached
  + charset-normalizer                2.0.10  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + colorama                           0.4.4  pyh9f0ad1d_0        conda-forge/noarch     Cached
  + cryptography                      36.0.0  py39hf6deb26_0      pkgs/main/osx-64       Cached
  + docutils                          0.18.1  py39h6e9494a_0      conda-forge/osx-64     Cached
  + idna                                 3.3  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + imagesize                          1.3.0  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + jinja2                             3.0.3  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + libffi                             3.4.2  h0d85af4_5          conda-forge/osx-64     Cached
  + libzlib                           1.2.11  h9173be1_1013       conda-forge/osx-64     Cached
  + markupsafe                         2.0.1  py39h89e85a6_1      conda-forge/osx-64     Cached
  + ncurses                              6.3  he49afe7_0          conda-forge/osx-64     926 KB
  + openssl                            3.0.0  h0d85af4_2          conda-forge/osx-64     Cached
  + packaging                           21.3  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + pip                               21.3.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + pycparser                           2.21  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + pydata-sphinx-theme                0.8.0  pyhd8ed1ab_0        conda-forge/noarch       1 MB
  + pygments                          2.11.2  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + pyopenssl                         21.0.0  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + pyparsing                          3.0.7  pyhd8ed1ab_0        conda-forge/noarch      79 KB
  + pysocks                            1.7.1  py39h6e9494a_4      conda-forge/osx-64     Cached
  + python                             3.9.9  h38b4d05_0_cpython  conda-forge/osx-64     Cached
  + python_abi                           3.9  2_cp39              conda-forge/osx-64     Cached
  + pytz                              2021.3  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + readline                             8.1  h05e3726_0          conda-forge/osx-64     Cached
  + requests                          2.27.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + setuptools                        60.5.0  py39h6e9494a_0      conda-forge/osx-64       1 MB
  + six                               1.16.0  pyh6c4a22f_0        conda-forge/noarch     Cached
  + snowballstemmer                    2.2.0  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + soupsieve                          2.3.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + sphinx                             3.5.3  pyhd3eb1b0_0        pkgs/main/noarch       Cached
  + sphinxcontrib-applehelp            1.0.2  py_0                conda-forge/noarch     Cached
  + sphinxcontrib-devhelp              1.0.2  py_0                conda-forge/noarch     Cached
  + sphinxcontrib-htmlhelp             2.0.0  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + sphinxcontrib-jsmath               1.0.1  py_0                conda-forge/noarch     Cached
  + sphinxcontrib-qthelp               1.0.3  py_0                conda-forge/noarch     Cached
  + sphinxcontrib-serializinghtml      1.1.5  pyhd8ed1ab_1        conda-forge/noarch     Cached
  + sqlite                            3.37.0  h23a322b_0          conda-forge/osx-64     Cached
  + tk                                8.6.11  h5dbffcc_1          conda-forge/osx-64     Cached
  + tzdata                             2021e  he74cb21_0          conda-forge/noarch     Cached
  + urllib3                           1.26.8  pyhd8ed1ab_1        conda-forge/noarch     Cached
  + wheel                             0.37.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + xz                                 5.2.5  haf1e3a3_1          conda-forge/osx-64     Cached
  + zlib                              1.2.11  h9173be1_1013       conda-forge/osx-64     Cached

[astrojuanlu]

I can reproduce your result. Both (sphinx 3.5, docutils 0.18) and (sphinx 4.x, docutils 0.17) are solutions to the dependency satisfaction problem, and I suppose that mamba does not have any information to favor one or another to prefer the most recent version. Doing mamba create -n testsphinx python=3.9 "sphinx>=4" pydata-sphinx-theme -c conda-forge (i.e. adding "sphinx>=4" to the mix) downgrades docutils as expected.