MAINT: Update test suite for Python 3.11-3.13 and Sphinx 6-8 compatibility (#79)

* Update test suite for Python 3.11-3.13 and Sphinx 6-8 compatibility

- Update pyproject.toml: require Python >=3.11, Sphinx >=6.1, matplotlib >=3.8
- Update tox.ini: add 9 test environments (py311/312/313 × sphinx6/7/8)
- Fix deprecation warnings:
  - Use os.path.splitext() instead of string operations (RemovedInSphinx90Warning)
  - Update sphinx.util.console import path
- Implement Sphinx-version-specific regression fixtures:
  - Generate separate .sphinx6/.sphinx7/.sphinx8 fixtures for HTML and XML
  - Track actual Sphinx output differences without filtering
  - Add ipykernel process ID normalization for test stability
- All 110 tests pass across all 9 Python/Sphinx combinations

* Fix MatplotlibDeprecationWarning in test code and update CHANGELOG

- Update CHANGELOG.md with v1.0.0 and v1.0.1 release information
- Fix deprecated matplotlib.axes.Axes.cohere() API usage in test files
  - Change from positional args to keyword args (NFFT=, Fs=)
  - Eliminates MatplotlibDeprecationWarning from test output
- Regenerate Sphinx 8 test fixtures to reflect corrected test code
- All 110 tests passing with only external myst_nb warnings remaining

* Update myst-nb dependency to >=1.1.0 to eliminate deprecation warnings

- Change from myst-nb~=1.0.0 to myst-nb>=1.1.0
- myst-nb v1.1.0+ uses Node.findall() instead of deprecated Node.traverse()
- Eliminates PendingDeprecationWarning and RemovedInSphinx90Warning
- All 110 tests pass with ZERO warnings

* Regenerate Sphinx 6 and 7 test fixtures with updated myst-nb

- Update fixtures to reflect myst-nb >=1.1.0 output (no warnings)
- Remove MatplotlibDeprecationWarning from fixture files

* Update pytest dependency from ~=8.0.0 to >=8.0

- Similar to myst-nb update, the ~=8.0.0 constraint was locking pytest to 8.0.x
- Latest pytest is 8.4.2 with bug fixes and improvements
- All 110 tests passing with pytest 8.4.2

* Apply pre-commit formatting fixes

- Fix E402: Move sphinx.locale import to top of file in nodes.py
- Add newline at end of MANIFEST.in
- Apply ruff-format line wrapping for long lines

* Normalize matplotlib image hashes in tests

Matplotlib generates platform/version-specific content hashes for images.
This causes test failures when comparing fixtures generated on different
platforms (macOS vs Linux) or with different package versions.

Solution: Add regex pattern to FileRegression to normalize all matplotlib
image hashes to 'IMAGEHASH.png', making tests platform-independent.

Fixes CI test failures in test_gateddirective.py.

* Update CI to test all Python and Sphinx combinations

Match CI test matrix to tox configuration:
- Python versions: 3.11, 3.12, 3.13
- Sphinx versions: 6, 7, 8
- Total: 9 test combinations

This ensures CI tests the same environments as local tox testing,
providing better coverage and catching version-specific issues.

* Add Sphinx version compatibility for strip_escape_sequences

The function strip_escape_sequences was removed from sphinx.util.console
in later versions. Add try/except import with fallback implementation.

Also update tox.ini to properly install Sphinx version-specific dependencies
by explicitly listing all testing dependencies in deps rather than using
the 'testing' extra which includes a broad Sphinx version range.

Fixes CI failure on Python 3.13 + Sphinx 6 combination.

* Fix tox to properly install version-specific Sphinx

Use constrain_package_deps and use_frozen_constraints to ensure
that the Sphinx version specified in tox deps takes precedence over
the package's own dependency declaration (sphinx>=6.1).

This creates a constraints file that locks the Sphinx version during
package dependency installation, preventing pip from upgrading to the
latest version.

Verified working:
- py311-sphinx6: Sphinx 6.2.1
- py311-sphinx7: Sphinx 7.4.7
- py311-sphinx8: Sphinx 8.2.3

* Simplify tox configuration by using testing extras

Now that constrain_package_deps is working correctly, we can use
the 'testing' extras from pyproject.toml instead of manually listing
all dependencies.

This is more maintainable - when testing dependencies change in
pyproject.toml, they automatically apply to tox without needing
to update both files.

The constraints mechanism ensures Sphinx version from deps still
takes precedence over the package's sphinx>=6.1,<9 requirement.

* Fix Sphinx 6 compatibility in conftest.py rootdir fixture

The sphinx.testing.path.path object in Sphinx <7.2 doesn't have an
.absolute() method. The path is already absolute, so we can use it
directly without calling .absolute().

This fixes the AttributeError: 'path' object has no attribute 'absolute'
error that was causing all 110 tests to fail on Sphinx 6.

The version check ensures Sphinx 7.2+ continues to use pathlib.Path
with .absolute() as before.

* Regenerate Sphinx 7 fixtures with normalized image hashes

* Add Sphinx 6 fixture files for complete test coverage

* Replace os.path.splitext with pathlib.Path for consistency

Use pathlib.Path.with_suffix('') instead of os.path.splitext()[0] for
better consistency with modern Python practices. This removes the os
import which was only used for this single operation.

* Replace remaining os.path usage with pathlib across codebase

Modernize all remaining os.path operations to use pathlib.Path for
consistency:
- sphinx_exercise/__init__.py: Use Path for package/locale directory
- sphinx_exercise/post_transforms.py: Use Path.with_suffix('')
- sphinx_exercise/translations/_convert.py: Use Path.resolve()

This completes the migration to pathlib throughout the codebase,
keeping only os.sep in tests where it's appropriate for path
separator normalization.

Author: mmcky

.github/workflows/ci.yml

@@ -23,7 +23,8 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.11", "3.12"]
+        python-version: ["3.11", "3.12", "3.13"]
+        sphinx-version: ["6", "7", "8"]
     steps:
       - uses: actions/checkout@v4
       - name: Set up Python ${{ matrix.python-version }}
@@ -34,16 +35,17 @@ jobs:
         run : |
           python -m pip install --upgrade pip
           pip install -e.[testing]
+          pip install "sphinx>=${{ matrix.sphinx-version }},<${{ matrix.sphinx-version == '6' && '7' || matrix.sphinx-version == '7' && '8' || '9' }}"
       - name: Run pytest
         run: |
           pytest --durations=10 --cov=sphinx_exercise --cov-report=xml --cov-report=term-missing
       - name: Create cov
         run: coverage xml
       - name: Upload to Codecov
-        if: false && (matrix.python-version == '3.11')
+        if: false && (matrix.python-version == '3.11' && matrix.sphinx-version == '8')
         uses: codecov/codecov-action@v4
         with:
-          name: sphinx-exercise-pytest-py3.11
+          name: sphinx-exercise-pytest-py3.11-sphinx8
           token: "${{ secrets.CODECOV_TOKEN }}"
           flags: pytests
           file: ./coverage.xml

CHANGELOG.md

@@ -1,6 +1,25 @@
 # Changelog
 
 
+## [v1.0.1](https://github.com/executablebooks/sphinx-exercise/tree/v1.0.1) (2024-07-01)
+
+### Improved 👌
+
+- Updates to testing infrastructure and minor bug fixes
+- Fixed JupyterBuilder handling in test suite
+
+## [v1.0.0](https://github.com/executablebooks/sphinx-exercise/tree/v1.0.0) (2024-01-15)
+
+### Improved 👌
+
+- Added support for Sphinx 7
+- Updated build and CI infrastructure
+- Fixed deprecation warnings for Sphinx 7 compatibility
+- Improved type hints throughout codebase
+- Fixed doctree node mutation issues by copying nodes
+- Updated codecov to version 3
+- Pinned matplotlib to 3.7.* for testing stability
+
 ## [v0.4.1](https://github.com/executablebooks/sphinx-exercise/tree/v0.4.1) (2023-1-23)
 
 ### Improved 👌

MANIFEST.in

@@ -8,4 +8,4 @@ recursive-include sphinx_exercise *.css
 recursive-include sphinx_exercise *.json
 recursive-include sphinx_exercise *.mo
 recursive-include sphinx_exercise *.po
-recursive-include sphinx_exercise *.py
+recursive-include sphinx_exercise *.py

pyproject.toml

@@ -8,7 +8,7 @@ dynamic = ["version"]
 description = "A Sphinx extension for producing exercises and solutions."
 readme = "README.md"
 license = { file = "LICENSE" }
-requires-python = ">=3.9"
+requires-python = ">=3.11"
 authors = [
     { name = "QuantEcon", email = "[email protected]" },
 ]
@@ -22,10 +22,9 @@ classifiers = [
     "Natural Language :: English",
     "Operating System :: OS Independent",
     "Programming Language :: Python",
-    "Programming Language :: Python :: 3.9",
-    "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
     "Topic :: Documentation",
     "Topic :: Documentation :: Sphinx",
     "Topic :: Software Development :: Documentation",
@@ -34,7 +33,7 @@ classifiers = [
 ]
 dependencies = [
     "sphinx-book-theme",
-    "sphinx>=5",
+    "sphinx>=6.1",
 ]
 
 [project.optional-dependencies]
@@ -49,20 +48,20 @@ code_style = [
     "pre-commit",
 ]
 rtd = [
-    "myst-nb~=1.0.0",
+    "myst-nb>=1.1.0",
     "sphinx-book-theme",
     "sphinx_togglebutton",
-    "sphinx>=5,<8",
+    "sphinx>=6.1,<9",
 ]
 testing = [
     "beautifulsoup4",
     "coverage",
-    "matplotlib==3.8.*",
-    "myst-nb~=1.0.0",
+    "matplotlib>=3.8",
+    "myst-nb>=1.1.0",
     "pytest-cov",
     "pytest-regressions",
-    "pytest~=8.0.0",
-    "sphinx>=5,<8",
+    "pytest>=8.0",
+    "sphinx>=6.1,<9",
     "texsoup",
     "defusedxml", # Required by sphinx-testing
 ]

sphinx_exercise/__init__.py

@@ -9,7 +9,6 @@
 
 __version__ = "1.0.1"
 
-import os
 from pathlib import Path
 from typing import Any, Dict, Set, Union, cast
 from sphinx.config import Config
@@ -71,6 +70,7 @@
 
 # Callback Functions
 
+
 def purge_exercises(app: Sphinx, env: BuildEnvironment, docname: str) -> None:
     """Purge sphinx_exercise registry"""
 
@@ -214,9 +214,9 @@ def setup(app: Sphinx) -> Dict[str, Any]:
     app.add_css_file("exercise.css")
 
     # add translations
-    package_dir = os.path.abspath(os.path.dirname(__file__))
-    locale_dir = os.path.join(package_dir, "translations", "locales")
-    app.add_message_catalog(MESSAGE_CATALOG_NAME, locale_dir)
+    package_dir = Path(__file__).parent.resolve()
+    locale_dir = package_dir / "translations" / "locales"
+    app.add_message_catalog(MESSAGE_CATALOG_NAME, str(locale_dir))
 
     return {
         "version": "builtin",

sphinx_exercise/directive.py

@@ -8,28 +8,30 @@
 :licences: see LICENSE for details
 """
 
+from pathlib import Path
 from typing import List
-from docutils.nodes import Node
 
-from sphinx.util.docutils import SphinxDirective
+from docutils import nodes
+from docutils.nodes import Node
 from docutils.parsers.rst import directives
+from sphinx.locale import get_translation
+from sphinx.util import logging
+from sphinx.util.docutils import SphinxDirective
+
 from .nodes import (
-    exercise_node,
-    exercise_enumerable_node,
     exercise_end_node,
+    exercise_enumerable_node,
+    exercise_node,
+    exercise_subtitle,
+    exercise_title,
+    solution_end_node,
     solution_node,
     solution_start_node,
-    solution_end_node,
-    exercise_title,
-    exercise_subtitle,
     solution_title,
 )
-from docutils import nodes
-from sphinx.util import logging
 
 logger = logging.getLogger(__name__)
 
-from sphinx.locale import get_translation
 MESSAGE_CATALOG_NAME = "exercise"
 translate = get_translation(MESSAGE_CATALOG_NAME)
 
@@ -40,7 +42,7 @@ def duplicate_labels(self, label):
 
         if not label == "" and label in self.env.sphinx_exercise_registry.keys():
             docpath = self.env.doc2path(self.env.docname)
-            path = docpath[: docpath.rfind(".")]
+            path = str(Path(docpath).with_suffix(""))
             other_path = self.env.doc2path(
                 self.env.sphinx_exercise_registry[label]["docname"]
             )

sphinx_exercise/nodes.py

@@ -13,13 +13,12 @@
 from docutils import nodes as docutil_nodes
 from sphinx import addnodes as sphinx_nodes
 from sphinx.writers.latex import LaTeXTranslator
+from sphinx.locale import get_translation
 from .latex import LaTeXMarkup
 
 logger = logging.getLogger(__name__)
 LaTeX = LaTeXMarkup()
 
-
-from sphinx.locale import get_translation
 MESSAGE_CATALOG_NAME = "exercise"
 translate = get_translation(MESSAGE_CATALOG_NAME)
 
@@ -54,7 +53,10 @@ class solution_end_node(docutil_nodes.Admonition, docutil_nodes.Element):
 class exercise_title(docutil_nodes.title):
     def default_title(self):
         title_text = self.children[0].astext()
-        if title_text == f"{translate('Exercise')}" or title_text == f"{translate('Exercise')} %s":
+        if (
+            title_text == f"{translate('Exercise')}"
+            or title_text == f"{translate('Exercise')} %s"
+        ):
             return True
         else:
             return False

sphinx_exercise/post_transforms.py

@@ -1,3 +1,5 @@
+from pathlib import Path
+
 import sphinx.addnodes as sphinx_nodes
 from sphinx.transforms.post_transforms import SphinxPostTransform
 from sphinx.util import logging
@@ -195,7 +197,7 @@ def run(self):
                     except AttributeError:
                         docname = self.env.docname  # for builder such as JupyterBuilder that don't support current_docname
                 docpath = self.env.doc2path(docname)
-                path = docpath[: docpath.rfind(".")]
+                path = str(Path(docpath).with_suffix(""))
                 msg = f"undefined label: {target_label}"
                 logger.warning(msg, location=path, color="red")
                 return

sphinx_exercise/translations/_convert.py

@@ -1,10 +1,10 @@
 import json
-import os
 from pathlib import Path
 import subprocess
 
 MESSAGE_CATALOG_NAME = "exercise"
 
+
 def convert_json(folder=None):
     folder = folder or Path(__file__).parent
 
@@ -19,7 +19,13 @@ def convert_json(folder=None):
         english = data[0]["text"]
         for item in data[1:]:
             language = item["symbol"]
-            out_path = folder / "locales" / language / "LC_MESSAGES" / f"{MESSAGE_CATALOG_NAME}.po"
+            out_path = (
+                folder
+                / "locales"
+                / language
+                / "LC_MESSAGES"
+                / f"{MESSAGE_CATALOG_NAME}.po"
+            )
             if not out_path.parent.exists():
                 out_path.parent.mkdir(parents=True)
             if not out_path.exists():
@@ -47,9 +53,9 @@ def convert_json(folder=None):
         subprocess.check_call(
             [
                 "msgfmt",
-                os.path.abspath(path),
+                str(path.resolve()),
                 "-o",
-                os.path.abspath(path.parent / f"{MESSAGE_CATALOG_NAME}.mo"),
+                str((path.parent / f"{MESSAGE_CATALOG_NAME}.mo").resolve()),
             ]
         )
 

tests/books/test-gateddirective/solution-exercise-gated.md

@@ -42,7 +42,7 @@ axs[0].set_xlabel('time')
 axs[0].set_ylabel('s1 and s2')
 axs[0].grid(True)
 
-cxy, f = axs[1].cohere(s1, s2, 256, 1. / dt)
+cxy, f = axs[1].cohere(s1, s2, NFFT=256, Fs=1. / dt)
 axs[1].set_ylabel('coherence')
 
 fig.tight_layout()
@@ -87,7 +87,7 @@ axs[0].set_xlabel('time')
 axs[0].set_ylabel('s1 and s2')
 axs[0].grid(True)
 
-cxy, f = axs[1].cohere(s1, s2, 256, 1. / dt)
+cxy, f = axs[1].cohere(s1, s2, NFFT=256, Fs=1. / dt)
 axs[1].set_ylabel('coherence')
 
 fig.tight_layout()