feat: support pytest 9's new config (#681)

* feat: support pytest 9's new config

Signed-off-by: Henry Schreiner <[email protected]>

* Apply suggestions from code review

Co-authored-by: Copilot <[email protected]>

---------

Signed-off-by: Henry Schreiner <[email protected]>
Co-authored-by: Copilot <[email protected]>

Author: henryiii

README.md

@@ -329,10 +329,10 @@ for family, grp in itertools.groupby(collected.checks.items(), key=lambda x: x[1
 - [`PP005`](https://learn.scientific-python.org/development/guides/packaging-simple#PP005): Using SPDX project.license should not use deprecated trove classifiers
 - [`PP006`](https://learn.scientific-python.org/development/guides/packaging-simple#PP006): The dev dependency group should be defined
 - [`PP301`](https://learn.scientific-python.org/development/guides/pytest#PP301): Has pytest in pyproject
-- [`PP302`](https://learn.scientific-python.org/development/guides/pytest#PP302): Sets a minimum pytest to at least 6
+- [`PP302`](https://learn.scientific-python.org/development/guides/pytest#PP302): Sets a minimum pytest to at least 6 or 9
 - [`PP303`](https://learn.scientific-python.org/development/guides/pytest#PP303): Sets the test paths
 - [`PP304`](https://learn.scientific-python.org/development/guides/pytest#PP304): Sets the log level in pytest
-- [`PP305`](https://learn.scientific-python.org/development/guides/pytest#PP305): Specifies xfail_strict
+- [`PP305`](https://learn.scientific-python.org/development/guides/pytest#PP305): Specifies strict xfail
 - [`PP306`](https://learn.scientific-python.org/development/guides/pytest#PP306): Specifies strict config
 - [`PP307`](https://learn.scientific-python.org/development/guides/pytest#PP307): Specifies strict markers
 - [`PP308`](https://learn.scientific-python.org/development/guides/pytest#PP308): Specifies useful pytest summary

docs/pages/guides/pytest.md

@@ -66,9 +66,12 @@ This looks simple, but it is doing several things:
 ### Configuring pytest
 
 pytest supports configuration in `pytest.ini`, `setup.cfg`, or, since version 6,
-`pyproject.toml` {% rr PP301 %}. Remember, pytest is a developer requirement,
-not a user one, so always require 6+ (or 7+) and use `pyproject.toml`. This is
-an example configuration:
+`pyproject.toml` {% rr PP301 %}, or, since version 9, `pytest.toml` or
+`.pytest.toml`. Remember, pytest is a developer requirement, not a user one, so
+always require 6+ (or 9+) and use `pyproject.toml` or the pytest TOML ones. This
+is an example configuration:
+
+{% tabs %} {% tab classic "Pytest 6+" %}
 
 ```toml
 [tool.pytest.ini_options]
@@ -82,23 +85,40 @@ testpaths = [
 ]
 ```
 
+{% endtab %} {% tab modern "Pytest 9+" %}
+
+```toml
+[tool.pytest]
+minversion = "9.0"
+addopts = ["-ra", "--showlocals"]
+strict = true
+filterwarnings = ["error"]
+log_level = "INFO"
+testpaths = [
+  "tests",
+]
+```
+
+{% endtab %} {% endtabs %}
+
 {% rr PP302 %} The `minversion` will print a nicer error if your `pytest` is too
 old (though, ironically, it won't read this if the version is too old, so
-setting "6" or less in `pyproject.toml` is rather pointless). The `addopts`
-setting will add whatever you put there to the command line when you run;
-{% rr PP308 %} `-ra` will print a summary "r"eport of "a"ll results, which gives
-you a quick way to review what tests failed and were skipped, and why.
-`--showlocals` will print locals in tracebacks. {% rr PP307 %}
-`--strict-markers` will make sure you don't try to use an unspecified fixture.
-{% rr PP306 %} And `--strict-config` will error if you make a mistake in your
-config. {% rr PP305 %} `xfail_strict` will change the default for `xfail` to
-fail the tests if it doesn't fail - you can still override locally in a specific
-xfail for a flaky failure. {% rr PP309 %} `filter_warnings` will cause all
-warnings to be errors (you can add allowed warnings here too, see below).
-{% rr PP304 %} `log_level` will report `INFO` and above log messages on a
-failure. {% rr PP303 %} Finally, `testpaths` will limit `pytest` to just looking
-in the folders given - useful if it tries to pick up things that are not tests
-from other directories.
+setting "6" or less in `pyproject.toml` is rather pointless, similarly for 9 if
+using the new config location). The `addopts` setting will add whatever you put
+there to the command line when you run; {% rr PP308 %} `-ra` will print a
+summary "r"eport of "a"ll results, which gives you a quick way to review what
+tests failed and were skipped, and why. `--showlocals` will print locals in
+tracebacks - depending on your tests, you might or might not like this one.
+{% rr PP307 %} `--strict-markers` will make sure you don't try to use an
+unspecified fixture. {% rr PP306 %} And `--strict-config` will error if you make
+a mistake in your config. {% rr PP305 %} `xfail_strict` will change the default
+for `xfail` to fail the tests if it doesn't fail - you can still override
+locally in a specific xfail for a flaky failure. {% rr PP309 %}
+`filter_warnings` will cause all warnings to be errors (you can add allowed
+warnings here too, see below). {% rr PP304 %} `log_level` will report `INFO` and
+above log messages on a failure. {% rr PP303 %} Finally, `testpaths` will limit
+`pytest` to just looking in the folders given - useful if it tries to pick up
+things that are not tests from other directories.
 [See the docs](https://docs.pytest.org/en/stable/customize.html) for more
 options.
 

pyproject.toml

@@ -70,13 +70,14 @@ setupcfg = "sp_repo_review.checks.setupcfg:repo_review_checks"
 noxfile = "sp_repo_review.checks.noxfile:repo_review_checks"
 
 [project.entry-points."repo_review.fixtures"]
-workflows = "sp_repo_review.checks.github:workflows"
 dependabot = "sp_repo_review.checks.github:dependabot"
+noxfile = "sp_repo_review.checks.noxfile:noxfile"
 precommit = "sp_repo_review.checks.precommit:precommit"
+pytest = "sp_repo_review.checks.pyproject:pytest"
 readthedocs = "sp_repo_review.checks.readthedocs:readthedocs"
 ruff = "sp_repo_review.checks.ruff:ruff"
 setupcfg = "sp_repo_review.checks.setupcfg:setupcfg"
-noxfile = "sp_repo_review.checks.noxfile:noxfile"
+workflows = "sp_repo_review.checks.github:workflows"
 
 [project.entry-points."repo_review.families"]
 scikit-hep = "sp_repo_review.families:get_families"

src/sp_repo_review/checks/pyproject.py

@@ -1,12 +1,47 @@
 from __future__ import annotations
 
+import enum
 from typing import TYPE_CHECKING, Any
 
+from .._compat import tomllib
 from . import mk_url
 
 if TYPE_CHECKING:
     from configparser import ConfigParser
 
+    from .._compat.importlib.resources.abc import Traversable
+
+
+class PytestFile(enum.Enum):
+    PYTEST_TOML = enum.auto()
+    MODERN_PYPROJECT = enum.auto()
+    LEGACY_PYPROJECT = enum.auto()
+    NONE = enum.auto()
+
+
+def pytest(
+    pyproject: dict[str, Any], root: Traversable
+) -> tuple[PytestFile, dict[str, Any]]:
+    """
+    Returns a tuple containing the pytest configuration file type and the configuration dictionary.
+    Returns (PytestFile.NONE, {}) if the configuration doesn't exist.
+    Respects toml configurations only.
+    """
+    paths = [root.joinpath("pytest.toml"), root.joinpath(".pytest.toml")]
+    for path in paths:
+        if path.is_file():
+            with path.open("rb") as f:
+                contents = tomllib.load(f)
+            return (PytestFile.PYTEST_TOML, contents.get("pytest", {}))
+
+    match pyproject:
+        case {"tool": {"pytest": {"ini_options": config}}}:
+            return (PytestFile.LEGACY_PYPROJECT, config)
+        case {"tool": {"pytest": config}}:
+            return (PytestFile.MODERN_PYPROJECT, config)
+        case _:
+            return (PytestFile.NONE, {})
+
 
 def get_requires_python(
     pyproject: dict[str, Any], setupcfg: ConfigParser | None
@@ -149,45 +184,45 @@ def check(pyproject: dict[str, Any]) -> bool:
 class PP301(PyProject):
     "Has pytest in pyproject"
 
-    requires = {"PY001"}
+    requires = set[str]()
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
-        Must have a `[tool.pytest.ini_options]` configuration section in
-        pyproject.toml. If you must have it somewhere else (such as to support
-        `pytest<6`), ignore this check.
+        Must have a `[tool.pytest]` (pytest 9+) or `[tool.pytest.ini_options]`
+        (pytest 6+) configuration section in pyproject.toml. If you must have it
+        in ini format, ignore this check.  pytest.toml and .pytest.toml files
+        (pytest 9+) are also supported.
         """
 
-        match pyproject:
-            case {"tool": {"pytest": {"ini_options": {}}}}:
-                return True
-            case _:
-                return False
+        loc, _ = pytest
+        return loc is not PytestFile.NONE
 
 
 class PP302(PyProject):
-    "Sets a minimum pytest to at least 6"
+    "Sets a minimum pytest to at least 6 or 9"
 
     requires = {"PP301"}
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
         Must have a `minversion=`, and must be at least 6 (first version to
-        support `pyproject.toml` configuration).
+        support `pyproject.toml` ini configuration) or 9 (first version to
+        support native configuration and toml config files).
 
         ```toml
         [tool.pytest.ini_options]
-        minversion = "7"
+        minversion = "9"
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
+        loc, options = pytest
+        minversion = 6 if loc is PytestFile.LEGACY_PYPROJECT else 9
         return (
             "minversion" in options
-            and int(str(options["minversion"]).split(".", maxsplit=1)[0]) >= 6
+            and int(str(options["minversion"]).split(".", maxsplit=1)[0]) >= minversion
         )
 
 
@@ -198,7 +233,7 @@ class PP303(PyProject):
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
         The `testpaths` setting should be set to a reasonable default.
 
@@ -207,7 +242,7 @@ def check(pyproject: dict[str, Any]) -> bool:
         testpaths = ["tests"]
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
+        _, options = pytest
         return "testpaths" in options
 
 
@@ -218,7 +253,7 @@ class PP304(PyProject):
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
         `log_level` should be set. This will allow logs to be displayed on
         failures.
@@ -228,29 +263,34 @@ def check(pyproject: dict[str, Any]) -> bool:
         log_level = "INFO"
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
+        _, options = pytest
         return "log_cli_level" in options or "log_level" in options
 
 
 class PP305(PyProject):
-    "Specifies xfail_strict"
+    "Specifies strict xfail"
 
     requires = {"PP301"}
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
-        `xfail_strict` should be set. You can manually specify if a check
-        should be strict when setting each xfail.
+        `xfail_strict`, or if using pytest 9+, `strict_xfail` or `strict` should
+        be set. You can manually specify if a check should be strict when
+        setting each xfail.
 
         ```toml
         [tool.pytest.ini_options]
         xfail_strict = true
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
-        return "xfail_strict" in options
+        _, options = pytest
+        return (
+            "xfail_strict" in options
+            or "strict_xfail" in options
+            or "strict" in options
+        )
 
 
 class PP306(PyProject):
@@ -260,18 +300,23 @@ class PP306(PyProject):
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
-        `--strict-config` should be in `addopts = [...]`. This forces an error
-        if a config setting is misspelled.
+        `--strict-config` should be in `addopts = [...]` or (pytest 9+)
+        `strict_config` or `strict` should be set. This forces an error if a
+        config setting is misspelled.
 
         ```toml
         [tool.pytest.ini_options]
         addopts = ["-ra", "--strict-config", "--strict-markers"]
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
-        return "--strict-config" in options.get("addopts", [])
+        _, options = pytest
+        return (
+            "strict" in options
+            or "strict_config" in options
+            or "--strict-config" in options.get("addopts", [])
+        )
 
 
 class PP307(PyProject):
@@ -281,18 +326,23 @@ class PP307(PyProject):
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
-        `--strict-markers` should be in `addopts = [...]`. This forces all
-        markers to be specified in config, avoiding misspellings.
+        `--strict-markers` should be in `addopts = [...]` or (pytest 9+)
+        `strict_markers` or `strict` should be set. This forces test markers to
+        be specified in config, avoiding misspellings.
 
         ```toml
         [tool.pytest.ini_options]
         addopts = ["-ra", "--strict-config", "--strict-markers"]
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
-        return "--strict-markers" in options.get("addopts", [])
+        _, options = pytest
+        return (
+            "strict" in options
+            or "strict_markers" in options
+            or "--strict-markers" in options.get("addopts", [])
+        )
 
 
 class PP308(PyProject):
@@ -302,7 +352,7 @@ class PP308(PyProject):
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
         An explicit summary flag like `-ra` should be in `addopts = [...]`
         (print summary of all fails/errors).
@@ -312,9 +362,9 @@ def check(pyproject: dict[str, Any]) -> bool:
         addopts = ["-ra", "--strict-config", "--strict-markers"]
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
+        loc, options = pytest
         addopts = options.get("addopts", [])
-        if isinstance(addopts, str):
+        if loc is PytestFile.LEGACY_PYPROJECT and isinstance(addopts, str):
             addopts = addopts.split()
         return any(opt.startswith("-r") for opt in addopts)
 
@@ -326,7 +376,7 @@ class PP309(PyProject):
     url = mk_url("pytest")
 
     @staticmethod
-    def check(pyproject: dict[str, Any]) -> bool:
+    def check(pytest: tuple[PytestFile, dict[str, Any]]) -> bool:
         """
         `filterwarnings` must be set (probably to at least `["error"]`). Python
         will hide important warnings otherwise, like deprecations.
@@ -336,7 +386,7 @@ def check(pyproject: dict[str, Any]) -> bool:
         filterwarnings = ["error"]
         ```
         """
-        options = pyproject["tool"]["pytest"]["ini_options"]
+        _, options = pytest
         return "filterwarnings" in options