refactor/ci: drop Python 3.7, add Python 3.11 (#1662)

* drop Python 3.7, add Python 3.11
* update flake8 config in setup.cfg
* enable rasterio & fiona optional deps on windows
* disable vtk on Python 3.11 until published to PyPI
* use importlib.metadata instead of pkg_resources
* link to NEP 29 in DEVELOPER.md

Author: mwtoews

.github/workflows/benchmark.yml

@@ -12,7 +12,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ ubuntu-latest, macos-latest, windows-latest ]
-        python-version: [ 3.7, 3.8, 3.9, "3.10" ]
+        python-version: [ 3.8, 3.9, "3.10", "3.11" ]
         exclude:
           # avoid shutil.copytree infinite recursion bug
           # https://github.com/python/cpython/pull/17098
@@ -115,7 +115,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: 3.8
           cache: 'pip'
           cache-dependency-path: setup.cfg
 

.github/workflows/commit.yml

@@ -24,7 +24,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: 3.8
           cache: 'pip'
           cache-dependency-path: setup.cfg
 
@@ -57,7 +57,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: 3.8
           cache: 'pip'
           cache-dependency-path: setup.cfg
 
@@ -113,7 +113,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: 3.8
           cache: 'pip'
           cache-dependency-path: setup.cfg
 
@@ -149,7 +149,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ ubuntu-latest, macos-latest, windows-latest ]
-        python-version: [ 3.7, 3.8, 3.9, "3.10" ]
+        python-version: [ 3.8, 3.9, "3.10", "3.11" ]
         exclude:
           # avoid shutil.copytree infinite recursion bug
           # https://github.com/python/cpython/pull/17098

.github/workflows/examples.yml

@@ -12,7 +12,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ ubuntu-latest, macos-latest, windows-latest ]
-        python-version: [ 3.7, 3.8, 3.9, "3.10" ]
+        python-version: [ 3.8, 3.9, "3.10", "3.11" ]
         exclude:
           # avoid shutil.copytree infinite recursion bug
           # https://github.com/python/cpython/pull/17098

.github/workflows/regression.yml

@@ -12,7 +12,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ ubuntu-latest, macos-latest, windows-latest ]
-        python-version: [ 3.7, 3.8, 3.9, "3.10" ]
+        python-version: [ 3.8, 3.9, "3.10", "3.11" ]
         exclude:
           # avoid shutil.copytree infinite recursion bug
           # https://github.com/python/cpython/pull/17098

.github/workflows/release.yml

@@ -27,7 +27,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: 3.8
           cache: 'pip'
           cache-dependency-path: setup.cfg
 
@@ -214,7 +214,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: 3.8
           cache: 'pip'
           cache-dependency-path: setup.cfg
 
@@ -343,7 +343,7 @@ jobs:
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
-          python-version: 3.7
+          python-version: 3.8
           cache: 'pip'
           cache-dependency-path: setup.cfg
 
@@ -393,4 +393,4 @@ jobs:
           
           Updates the `develop` branch from `master` following a successful release. Increments the patch version number.
           '
-          gh pr create -B "develop" -H "$reset_branch" --title "Reinitialize develop branch" --draft --body "$body"
+          gh pr create -B "develop" -H "$reset_branch" --title "Reinitialize develop branch" --draft --body "$body"

DEVELOPER.md

@@ -47,7 +47,9 @@ GitHub's  [Guide to Installing Git](https://help.github.com/articles/set-up-git)
 
 ### Python
 
-Install Python 3.7.x or >=3.8.1, via [standalone download](https://www.python.org/downloads/) or a distribution like [Anaconda](https://www.anaconda.com/products/individual) or [miniconda](https://docs.conda.io/en/latest/miniconda.html). (An [infinite recursion bug](https://github.com/python/cpython/pull/17098) in 3.8.0's [`shutil.copytree`](https://github.com/python/cpython/commit/65c92c5870944b972a879031abd4c20c4f0d7981) can cause test failures if the destination is a subdirectory of the source.)
+FloPy supports several recent versions of Python, loosely following [NEP 29](https://numpy.org/neps/nep-0029-deprecation_policy.html#implementation).
+
+Install Python >=3.8.1, via [standalone download](https://www.python.org/downloads/) or a distribution like [Anaconda](https://www.anaconda.com/products/individual) or [miniconda](https://docs.conda.io/en/latest/miniconda.html). (An [infinite recursion bug](https://github.com/python/cpython/pull/17098) in 3.8.0's [`shutil.copytree`](https://github.com/python/cpython/commit/65c92c5870944b972a879031abd4c20c4f0d7981) can cause test failures if the destination is a subdirectory of the source.)
 
 Then install `flopy` and core dependencies from the project root:
 

README.md

@@ -30,7 +30,7 @@ For general modeling issues, please consult a modeling forum, such as the [MODFL
 Installation
 -----------------------------------------------
 
-FloPy requires **Python** 3.7 (or higher), **NumPy** 1.15.0 (or higher), and **matplotlib** 1.4.0 (or higher).  Dependencies for optional FloPy methods are summarized [here](docs/flopy_method_dependencies.md).
+FloPy requires **Python** 3.8 (or higher), **NumPy** 1.15.0 (or higher), and **matplotlib** 1.4.0 (or higher).  Dependencies for optional FloPy methods are summarized [here](docs/flopy_method_dependencies.md).
 
 To install FloPy type:
 

autotest/conftest.py

@@ -1,7 +1,9 @@
 import importlib
 import os
+import re
 import socket
 import sys
+from importlib import metadata
 from os import environ
 from os.path import basename, normpath
 from pathlib import Path
@@ -13,7 +15,6 @@
 from warnings import warn
 
 import matplotlib.pyplot as plt
-import pkg_resources
 import pytest
 from modflow_devtools.misc import is_in_ci
 
@@ -121,34 +122,40 @@ def pytest_addoption(parser):
 def pytest_report_header(config):
     """Header for pytest to show versions of packages."""
 
-    # if we ever drop support for python 3.7, could use importlib.metadata instead?
-    # or importlib_metadata backport: https://importlib-metadata.readthedocs.io/en/latest/
-    # pkg_resources discouraged: https://setuptools.pypa.io/en/latest/pkg_resources.html
+    required = []
+    extra = {}
+    for item in metadata.requires("flopy"):
+        pkg_name = re.findall(r"[a-z0-9_\-]+", item, re.IGNORECASE)[0]
+        if res := re.findall("extra == ['\"](.+)['\"]", item):
+            assert len(res) == 1, item
+            pkg_extra = res[0]
+            if pkg_extra not in extra:
+                extra[pkg_extra] = []
+            extra[pkg_extra].append(pkg_name)
+        else:
+            required.append(pkg_name)
 
     processed = set()
-    flopy_pkg = pkg_resources.get_distribution("flopy")
     lines = []
     items = []
-    for pkg in flopy_pkg.requires():
-        name = pkg.name
+    for name in required:
         processed.add(name)
         try:
-            version = pkg_resources.get_distribution(name).version
+            version = metadata.version(name)
             items.append(f"{name}-{version}")
-        except pkg_resources.DistributionNotFound:
+        except metadata.PackageNotFoundError:
             items.append(f"{name} (not found)")
     lines.append("required packages: " + ", ".join(items))
     installed = []
     not_found = []
-    for pkg in flopy_pkg.requires(["optional"]):
-        name = pkg.name
+    for name in extra["optional"]:
         if name in processed:
             continue
         processed.add(name)
         try:
-            version = pkg_resources.get_distribution(name).version
+            version = metadata.version(name)
             installed.append(f"{name}-{version}")
-        except pkg_resources.DistributionNotFound:
+        except metadata.PackageNotFoundError:
             not_found.append(name)
     if installed:
         lines.append("optional packages: " + ", ".join(installed))

etc/environment.yml

@@ -3,7 +3,7 @@ channels:
   - conda-forge
 dependencies:
   # required
-  - python>=3.7
+  - python>=3.8
   - numpy>=1.15.0
   - matplotlib>=1.4.0
 

setup.cfg

@@ -22,10 +22,10 @@ classifiers =
     Operating System :: MacOS
     Programming Language :: Python
     Programming Language :: Python :: 3
-    Programming Language :: Python :: 3.7
     Programming Language :: Python :: 3.8
     Programming Language :: Python :: 3.9
     Programming Language :: Python :: 3.10
+    Programming Language :: Python :: 3.11
     Programming Language :: Python :: 3 :: Only
     Topic :: Scientific/Engineering :: Hydrology
 url = https://github.com/modflowpy/flopy
@@ -40,7 +40,7 @@ project_urls =
 include_package_data = True  # includes files listed in MANIFEST.in
 zip_safe = False
 packages = find:
-python_requires = >=3.7
+python_requires = >=3.8
 install_requires =
     numpy >=1.15.0
     matplotlib >=1.4.0
@@ -70,18 +70,18 @@ test =
 optional =
     affine
     descartes
-    fiona; platform_system!='Windows'
+    fiona
     geojson
     netcdf4
     pandas
     pyproj
     pyshp
     python-dateutil >=2.4.0
-    rasterio; platform_system!='Windows'
-    rasterstats; platform_system!='Windows'
+    rasterio
+    rasterstats
     scipy
     shapely >=1.8
-    vtk
+    vtk; python_version<'3.11'
     xmipy
 doc =
     %(optional)s
@@ -118,28 +118,22 @@ exclude =
     autotest
 ignore =
     # https://flake8.pycqa.org/en/latest/user/error-codes.html
-    F401 # 'module' imported but unused
+    F401,
     # https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes
-    E121 # continuation line under-indented for hanging indent
-    E122 # continuation line missing indentation or outdented
-    E126 # continuation line over-indented for hanging indent
-    E127 # continuation line over-indented for visual indent
-    E128 # continuation line under-indented for visual indent
-    E203 # whitespace before
-    E221 # multiple spaces before operator
-    E222 # multiple spaces after operator
-    E226 # missing whitespace around arithmetic operator
-    E231 # missing whitespace after ','
-    E241 # multiple spaces after ','
-    E402 # module level import not at top of file
-    E501 # line too long (> 79 characters)
-    E502 # backslash is redundant between brackets
-    E722 # do not use bare 'except'
-    E741 # ambiguous variable name
-    W291 # trailing whitespace
-    W292 # no newline at end of file
-    W293 # blank line contains whitespace
-    W391 # blank line at end of file
-    W503 # line break before binary operator
-    W504 # line break after binary operator
+    # Indentation
+    E121, E122, E126, E127, E128,
+    # Whitespace
+    E203, E221, E222, E226, E231, E241,
+    # Import
+    E402,
+    # Line length
+    E501, E502,
+    # Statement
+    E722, E741,
+    # Whitespace warning
+    W291, W292, W293,
+    # Blank line warning
+    W391,
+    # Line break warning
+    W503, W504
 statistics = True