MAINT: Updates for pytest 8.1 and python 3.12

Author: larsoner

.github/workflows/base.yml

@@ -14,10 +14,10 @@ jobs:
   list_nox_test_sessions:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v1
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: "3.7"
           architecture: x64
 
       - name: Install noxfile requirements
@@ -43,14 +43,14 @@ jobs:
     name: ${{ matrix.os }} ${{ matrix.nox_session }} # ${{ matrix.name_suffix }}
     runs-on: ${{ matrix.os }}
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
 
       # Conda install
-      - name: Install conda v3.7
+      - name: Install conda v3.12
         uses: conda-incubator/setup-miniconda@v2
         with:
           # auto-update-conda: true
-          python-version: 3.7
+          python-version: "3.12"
           activate-environment: noxenv
       - run: conda info
         shell: bash -l {0}  # so that conda works
@@ -84,7 +84,7 @@ jobs:
           GITHUB_CONTEXT: ${{ toJSON(github) }}
         run: echo "$GITHUB_CONTEXT"
 
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           fetch-depth: 0  # so that gh-deploy works
 
@@ -100,7 +100,7 @@ jobs:
         uses: conda-incubator/setup-miniconda@v2
         with:
           # auto-update-conda: true
-          python-version: 3.7
+          python-version: "3.12"
           activate-environment: noxenv
       - run: conda info
         shell: bash -l {0}  # so that conda works

README.md

@@ -31,11 +31,8 @@ You should then be able to list all available tasks using:
 >>> nox --list
 Sessions defined in <path>\noxfile.py:
 
-* tests-2.7 -> Run the test suite, including test reports generation and coverage reports.
-* tests-3.5 -> Run the test suite, including test reports generation and coverage reports.
-* tests-3.6 -> Run the test suite, including test reports generation and coverage reports.
+* tests-3.12 -> Run the test suite, including test reports generation and coverage reports.
 * tests-3.8 -> Run the test suite, including test reports generation and coverage reports.
-* tests-3.7 -> Run the test suite, including test reports generation and coverage reports.
 - docs-3.7 -> Generates the doc and serves it on a local http server. Pass '-- build' to build statically instead.
 - publish-3.7 -> Deploy the docs+reports on github pages. Note: this rebuilds the docs
 - release-3.7 -> Create a release on github corresponding to the latest tag
@@ -49,7 +46,7 @@ This project uses `pytest` so running `pytest` at the root folder will execute a
 nox
 ```
 
-Tests and coverage reports are automatically generated under `./docs/reports` for one of the sessions (`tests-3.7`). 
+Tests and coverage reports are automatically generated under `./docs/reports` for one of the sessions (`tests-3.7`).
 
 If you wish to execute tests on a specific environment, use explicit session names, e.g. `nox -s tests-3.6`.
 

ci_tools/nox_utils.py

@@ -21,7 +21,7 @@
 nox_logger = logging.getLogger("nox")
 
 
-PY27, PY35, PY36, PY37, PY38 = "2.7", "3.5", "3.6", "3.7", "3.8"
+PY38, PY312 = "3.8", "3.12"
 DONT_INSTALL = "dont_install"
 
 

docs/changelog.md

@@ -1,5 +1,9 @@
 # Changelog
 
+### 2.0.0 - updates for Python 3.8+
+
+- Modernized for Python 3.8+ support only.
+
 ### 1.10.4 - python 3.5 xdist bugfix
 
  - Fixed issue with `pytest-xdist` and python 3.5: `pathlib` objects were not properly handled by other stdlib modules in this python version. Fixed [#59](https://github.com/smarie/python-pytest-harvest/issues/59)
@@ -144,31 +148,31 @@ Fixed pytest ordering issue, by relying on [place_as](https://github.com/pytest-
 
 ### 1.0.0 - new methods for pytest session analysis
 
-New methods are provided to analyse pytest session results: 
+New methods are provided to analyse pytest session results:
  - `filter_session_items(session, filter=None)` is the filtering method used behind several functions in this package - it can be used independently. `pytest_item_matches_filter` is the inner method used to test if a single item matches the filter.
  - `get_all_pytest_param_names(session, filter=None, filter_incomplete=False)` lists all unique parameter names used in pytest session items, with optional filtering capabilities. Fixes [#12](https://github.com/smarie/python-pytest-harvest/issues/12)
- - `is_pytest_incomplete(item)`, `get_pytest_status(item)`, `get_pytest_param_names(item)` and `get_pytest_params(item)` allow users to analyse a specific item. 
+ - `is_pytest_incomplete(item)`, `get_pytest_status(item)`, `get_pytest_param_names(item)` and `get_pytest_params(item)` allow users to analyse a specific item.
 
 
 ### 0.9.0 - `get_session_synthesis_dct`: filter bugfix + test id formatter
 
  * `get_session_synthesis_dct`:
-  
+
    - `filter` now correctly handles class methods. Fixed [#11](https://github.com/smarie/python-pytest-harvest/issues/11)
    - new `test_id_format` option to process test ids. Fixed [#9](https://github.com/smarie/python-pytest-harvest/issues/9)
 
 ### 0.8.0 - Documentation + better filters in `get_session_synthesis_dct`
 
  * Documentation: added a section about creating the synthesis table from *inside* a test function (fixes [#4](https://github.com/smarie/python-pytest-harvest/issues/4)). Also, added a link to a complete example file.
- 
+
  * `get_session_synthesis_dct`: `filter` argument can now contain module names (fixed [#7](https://github.com/smarie/python-pytest-harvest/issues/7)). Also now the function filters out incomplete tests by default. A new `filter_incomplete` argument can be used to display them again (fixed [#8](https://github.com/smarie/python-pytest-harvest/issues/8)).
 
 ### 0.7.0 - Documentation + `get_session_synthesis_dct` improvements 2
 
  * Results bags do not measure execution time anymore since this is much less accurate than pytest duration. Fixes [#6](https://github.com/smarie/python-pytest-harvest/issues/6)
- 
+
  * `get_session_synthesis_dct` does not output the stage by stage details (setup/call/teardown) anymore by default, but a new option `status_details` allows users to enable them. Fixes [#5](https://github.com/smarie/python-pytest-harvest/issues/5)
-   
+
  * `get_session_synthesis_dct` has also 2 new options `durations_in_ms` and `pytest_prefix` to better control the output.
 
  * Improved documentation.

noxfile.py

@@ -9,7 +9,7 @@
 # add parent folder to python path so that we can import noxfile_utils.py
 # note that you need to "pip install -r noxfile-requiterements.txt" for this file to work.
 sys.path.append(str(Path(__file__).parent / "ci_tools"))
-from nox_utils import PY27, PY37, PY36, PY35, PY38, power_session, rm_folder, rm_file, PowerSession, DONT_INSTALL  # noqa
+from nox_utils import PY38, PY312, power_session, rm_folder, rm_file, PowerSession, DONT_INSTALL  # noqa
 
 
 pkg_name = "pytest_harvest"
@@ -18,35 +18,16 @@
 
 
 ENVS = {
-    # python 3.8 - put first to detect easy issues faster.
+    # python 3.12
+    (PY312, "pytest7.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<8"}},
+    (PY312, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
+    # python 3.8
     (PY38, "pytest2.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<3", "pytest-asyncio": DONT_INSTALL}},  # "pytest-html": "1.9.0",
     (PY38, "pytest3.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<4"}},
     (PY38, "pytest4.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<5"}},
     (PY38, "pytest5.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<6"}},
-    (PY38, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
-    # python 2.7
-    (PY27, "pytest2.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<3", "pytest-asyncio": DONT_INSTALL}},  # "pytest-html": "1.9.0",
-    (PY27, "pytest3.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<4"}},
-    (PY27, "pytest4.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<5"}},
-    # python 3.5
-    (PY35, "pytest2.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<3", "pytest-asyncio": DONT_INSTALL}},  # "pytest-html": "1.9.0",
-    (PY35, "pytest3.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<4"}},
-    (PY35, "pytest4.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<5"}},
-    (PY35, "pytest5.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<6"}},
-    (PY35, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
-    # python 3.6
-    (PY36, "pytest2.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<3", "pytest-asyncio": DONT_INSTALL}},  # "pytest-html": "1.9.0",
-    (PY36, "pytest3.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<4"}},
-    (PY36, "pytest4.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<5"}},
-    (PY36, "pytest5.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<6"}},
-    (PY36, "pytest-latest"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": ""}},
-    # python 3.7
-    (PY37, "pytest2.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<3", "pytest-asyncio": DONT_INSTALL}},  # "pytest-html": "1.9.0",
-    (PY37, "pytest3.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<4"}},
-    (PY37, "pytest4.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<5"}},
-    (PY37, "pytest5.x"): {"coverage": False, "pkg_specs": {"pip": ">19", "pytest": "<6"}},
     # IMPORTANT: this should be last so that the folder docs/reports is not deleted afterwards
-    (PY37, "pytest-latest"): {"coverage": True, "pkg_specs": {"pip": ">19", "pytest": ""}}
+    (PY38, "pytest-latest"): {"coverage": True, "pkg_specs": {"pip": ">19", "pytest": ""}}
 }
 
 
@@ -156,7 +137,7 @@ def tests(session: PowerSession, coverage, pkg_specs):
         session.run2("python ci_tools/generate-junit-badge.py 100 %s" % Folders.test_reports)
 
 
-@power_session(python=[PY37])
+@power_session(python=[PY38])
 def docs(session: PowerSession):
     """Generates the doc and serves it on a local http server. Pass '-- build' to build statically instead."""
 
@@ -169,7 +150,7 @@ def docs(session: PowerSession):
         session.run2("mkdocs serve -f ./docs/mkdocs.yml")
 
 
-@power_session(python=[PY37])
+@power_session(python=[PY38])
 def publish(session: PowerSession):
     """Deploy the docs+reports on github pages. Note: this rebuilds the docs"""
 
@@ -194,7 +175,7 @@ def publish(session: PowerSession):
     # session.run2('codecov -t %s -f %s' % (codecov_token, Folders.coverage_xml))
 
 
-@power_session(python=[PY37])
+@power_session(python=[PY38])
 def release(session: PowerSession):
     """Create a release on github corresponding to the latest tag"""
 

pytest_harvest/fixture_cache.py

@@ -3,7 +3,6 @@
 
 from decopatch import DECORATED, function_decorator
 from makefun import wraps, add_signature_parameters
-from six import string_types
 
 from pytest_harvest.common import get_scope
 
@@ -81,7 +80,7 @@ def test_synthesis(fixture_store):
     key = key or fixture_name
 
     # is the store a fixture or an object ?
-    store_is_a_fixture = isinstance(store, string_types)
+    store_is_a_fixture = isinstance(store, str)
 
     # if the store object is already available, we can ensure that it is initialized. Otherwise trust pytest for that
     if not store_is_a_fixture:

pytest_harvest/plugin.py

@@ -3,7 +3,6 @@
 from logging import warning
 from shutil import rmtree
 import pytest
-import six
 
 try:
     from pathlib import Path
@@ -238,9 +237,9 @@ def get_session_results_df(session_or_request,
     """
     try:
         import pandas as pd  # pylint: disable=import-outside-toplevel
-    except ImportError as e:
-        six.raise_from(Exception("There was an error importing `pandas` module. Fixture `session_results_df` and method"
-                                 "`get_session_results_df` can not be used in this session."), e)
+    except ImportError:
+        raise Exception("There was an error importing `pandas` module. Fixture `session_results_df` and method"
+                        "`get_session_results_df` can not be used in this session.")
 
     # in case of xdist, make sure persisted workers results have been reloaded
     possibly_restore_xdist_workers_structs(session_or_request)
@@ -308,10 +307,10 @@ def get_filtered_results_df(session,
     """
     try:
         import pandas as pd  # pylint: disable=import-outside-toplevel
-    except ImportError as e:
-        six.raise_from(Exception("There was an error importing `pandas` module. Fixture `session_results_df` and "
-                                 "methods `get_filtered_results_df` and `get_module_results_df` can not be used in this"
-                                 " session. "), e)
+    except ImportError:
+        raise Exception("There was an error importing `pandas` module. Fixture `session_results_df` and "
+                        "methods `get_filtered_results_df` and `get_module_results_df` can not be used in this"
+                        " session. ")
 
     # in case of xdist, make sure persisted workers results have been reloaded
     possibly_restore_xdist_workers_structs(session)

pytest_harvest/results_bags.py

@@ -1,7 +1,6 @@
 from datetime import datetime
 
 import pytest
-from six import raise_from
 
 try:  # python 3+
     from typing import Type, Set, Union, Any, Dict
@@ -25,19 +24,19 @@ def __setattr__(self, key, value):
         # try:  No exception can happen: key is always a string, and new entries are allowed in a dict
         self[key] = value
         # except KeyError as e:
-        #     raise_from(AttributeError(key), e)
+        #     raise (AttributeError(key)
 
     def __getattr__(self, key):
         try:
             return self[key]
         except KeyError as e:
-            raise_from(AttributeError(key), e)
+            raise AttributeError(key)
 
     def __delattr__(self, key):
         try:
             del self[key]
         except KeyError as e:
-            raise_from(AttributeError(key), e)
+            raise AttributeError(key)
 
     # object base
     def __str__(self):

pytest_harvest/results_session.py

@@ -1,13 +1,13 @@
-from distutils.version import LooseVersion
+from typing import Union, Iterable, Mapping, Any
+from packaging.version import parse
 
 import pytest
 import sys
 from collections import OrderedDict, namedtuple
 from itertools import chain
-from six import string_types
 
 
-pytest53 = LooseVersion(pytest.__version__) >= LooseVersion("5.3.0")
+pytest53 = parse(pytest.__version__) >= parse("5.3.0")
 if pytest53:
     def is_lazy_value_or_tupleitem_with_int_base(o):
         return False
@@ -24,15 +24,10 @@ def is_lazy_value_or_tupleitem_with_int_base(o):
         def is_lazy_value_or_tupleitem_with_int_base(o):
             return False
 
-try: # python 3.5+
-    from typing import Union, Iterable, Mapping, Any
-except ImportError:
-    pass
-
 from pytest_harvest.common import HARVEST_PREFIX
 from _pytest.doctest import DoctestItem
 
-
+pytest81 = parse(pytest.__version__) >= parse("8.1")
 PYTEST_OBJ_NAME = 'pytest_obj'
 
 
@@ -189,7 +184,7 @@ def test_id_format(test_id):
     if flatten_more is not None:
         if isinstance(flatten_more, dict):
             flatten_more_prefixes_dct = flatten_more.items()
-        elif isinstance(flatten_more, string_types):
+        elif isinstance(flatten_more, str):
             # single name ?
             flatten_more_prefixes_dct = {flatten_more: ''}
         else:
@@ -504,7 +499,9 @@ def get_pytest_params(item):
                     if is_lazy_value_or_tupleitem_with_int_base(param_value):
                         # remove the int base so that pandas does not interprete it as an int.
                         param_value = param_value.clone(remove_int_base=True)
-                    if item.session._fixturemanager.getfixturedefs(param_name, item.nodeid) is not None:
+
+                    arg = item if pytest81 else item.nodeid
+                    if item.session._fixturemanager.getfixturedefs(param_name, arg) is not None:
                         # Fixture parameters have the same name than the fixtures themselves! change it
                         param_dct[param_name + '_param'] = param_value
                     else:
@@ -543,7 +540,7 @@ def _get_filterset(filter):
     :param filter:
     :return:
     """
-    if isinstance(filter, string_types):
+    if isinstance(filter, str):
         filter = {filter}
     else:
         try:

pytest_harvest/tests/test_all_raw_with_meta_check.py

@@ -4,7 +4,6 @@
 from os.path import join, dirname, pardir
 
 import pytest
-import six
 
 # Make the list of all tests that we will have to execute (each in an independent pytest runner)
 THIS_DIR = dirname(__file__)
@@ -60,6 +59,6 @@ def test_run_all_tests(test_to_run, testdir):
         # Here we check that everything is ok
         try:
             result.assert_outcomes(**asserts_dct)
-        except Exception as e:
+        except Exception:
             print("Error while asserting that %s results in %s" % (test_to_run, str(asserts_dct)))
-            six.raise_from(e, e)
+            raise