style: run ruff lint and format on python files in repo (#1936)

* add ruff lint and format check to CI workflow.
* convert pyproject.toml to ruff.toml
* reduced number of ignored lint checks
* reformat remaining python files
* update CONTRIBUTING, DEVELOPER.md, and .vscode/README.md
* add vscode formatting tasks
* use flynt to convert .format() to f-strings

Author: jdhughes-dev

.doc/conf.py

@@ -88,7 +88,7 @@
     rtds_action_github_token = os.environ.get("GITHUB_TOKEN", None)
 
 # set master doc for readthedoce
-master_doc = 'index'
+master_doc = "index"
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]

.github/PULL_REQUEST_TEMPLATE.md

@@ -6,7 +6,7 @@ Checklist of items for pull request
 - [ ] Closed issue #xxxx
 - [ ] Referenced issue or pull request #xxxx
 - [ ] Added new test or modified an existing test
-- [ ] Ran `black` on new and modified autotests
+- [ ] Ran `ruff` on new and modified python scripts in .doc, autotests, doc, distribution, pymake, and utils subdirectories.
 - [ ] Formatted new and modified Fortran source files with `fprettify`
 - [ ] Added doxygen comments to new and modified procedures
 - [ ] Updated meson files, makefiles, and Visual Studio project files for new source files

.github/workflows/ci.yml

@@ -58,6 +58,12 @@ jobs:
 
       - name: Check MSVS project files
         run: pixi run check-vfproj
+
+      - name: Check python lint
+        run: pixi run check-python-lint
+
+      - name: Check python format
+        run: pixi run check-python-format
 
       - name: Check CITATION.cff
         uses: dieghernan/cff-validator@v3

.gitignore

@@ -70,10 +70,6 @@ bin/**
 UserPrefs.xml
 
 tmp_simulations/
-autotest/temp/
-autotest/tempbin/
-autotest/*.html
-autotest/notebooks/
 
 distribution/temp/
 distribution/temp_zip/
@@ -135,6 +131,11 @@ modflow6.code-workspace
 # unittests
 unittests/
 autotest/.failed
+autotest/.temp/
+autotest/temp/
+autotest/tempbin/
+autotest/*.html
+autotest/notebooks/
 
 **/__pycache__
 **/.benchmarks

.vscode/README.md

@@ -1,100 +1,129 @@
-# Compiling and Visual Debugging with VSCode
+# Using VSCode with MODFLOW 6
 
-VSCode is a free integrated development environment for Windows, Linux, and
-MacOS. There are two major advantages over Visual Studio:
+This document describes how to use VSCode to modify, format, build and test MODFLOW 6. VSCode is a free integrated development environment for Windows, Linux, and MacOS. Most of VSCode's source code is open-source and the releases are (proprietary) freeware. 
 
-* Most of VSCode's source code is open-source and the releases are (proprietary) freeware.
-* VSCode runs on Linux.
+The folder containing this README markdown file (`.vscode/`) contains the configuration files for using VSCode for MODFLOW 6 development. At the moment they are used for formatting, building, and debugging MODFLOW 6. In order to build MODFLOW 6 follow the steps in [DEVELOPER.md](../DEVELOPER.md)
 
-This folder contains the configuration files for using VSCode for MODFLOW 6 
-development.
-At the moment they are used for building and debugging MODFLOW 6.
-In order to build MODFLOW 6 follow the steps in [DEVELOPER.md](../DEVELOPER.md)
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
-## Visual Studio Code
+- [Setting up Visual Studio Code](#setting-up-visual-studio-code)
+  - [Installation](#installation)
+  - [Visual Studio Code Extensions](#visual-studio-code-extensions)
+  - [Dependencies](#dependencies)
+  - [Running VSCode](#running-vscode)
+  - [Final VSCode setup](#final-vscode-setup)
+- [MODFLOW 6 VSCode Tasks](#modflow-6-vscode-tasks)
+  - [Compiling](#compiling)
+  - [Debugging](#debugging)
+  - [Formatting](#formatting)
+  - [Additional tasks](#additional-tasks)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Setting up Visual Studio Code
+
+### Installation
 
 Install VSCode from https://code.visualstudio.com/
 
-### Extensions
+### Visual Studio Code Extensions
 
 Install the following VSCode extensions:
 
 - Modern Fortran:
   https://marketplace.visualstudio.com/items?itemName=krvajalm.linter-gfortran
 
-Note: The Remote - WSL extension may be required if you want to use a windows VSCode
-installation in a WSL environment.
+Note: The [Remote - WSL extension](https://code.visualstudio.com/docs/remote/wsl) may be required if you want to use a windows VSCode installation in a WSL environment.
 
 ### Dependencies
 
-Required and optional dependencies for MODFLOW 6 are discussed in [DEVELOPER.md](../DEVELOPER.md)
+Required and optional dependencies for MODFLOW 6 are discussed in [DEVELOPER.md](../DEVELOPER.md). Required dependencies should be installed prior to setting up VSCode for MODFLOW 6.
 
-### Settings
-
-Add [settings.json](https://code.visualstudio.com/docs/getstarted/settings#_settingsjson) to the
-`modflow6/.vscode` directory if not already there. The following settings can be considered a
-starting place as the contents of this file are dictated both by desired VSCode behavior and
-environmental factors:
+### Running VSCode
 
-In general, to determine the path of an installed tool in your environment run:
-- bash: `which <toolname>`, e.g. `which fortls`
-- cmd: `where <toolname>`, e.g. `where python`
-- PowerShell: `Get-Command <toolname>` e.g. `Get-Command fprettify`
+Open the top level `modflow6` repository directory on your system when starting VSCode. The program will then look for the `modflow6/.vscode` directory to discover settings relevant to your session.
 
-1. Activate the conda environment:
+A nice alternative on any system is to start VSCode from the shell. For example, in a bash or git bash shell (windows), change to the `modflow6` directory and execute the command:
 
 ```bash
-conda activate modflow6
+code .
 ```
 
-2. Determine the path of `fortls` and `fprettify`
+Note the dot (".") at the end of the command. Starting in this way, VSCode will open as desired, inheriting and discovering expected runtime settings in the correct directory location.
 
-3. Set the setting "fortran.fortls.path" and "fortran.formatting.path":
-```json
-{
-    "fortran.fortls.path": "/path/to/fortls",
-    "fortran.formatting.path": "/path/to/fprettify",
-}
-```
+### Final VSCode setup
 
-The fortran formatter can be integrated with VSCode using the following settings:
+A few additional steps are required to finalize the VSCode setup for MODFLOW 6. The remaining steps are:
 
-```json
-{
-    "[fortran]": {
-        "editor.formatOnSave": true,
-    },
-    "fortran.formatting.formatter": "fprettify",
-    "fortran.formatting.fprettifyArgs": ["-c", "/path/to/modflow6/.fprettify.yaml"],
-}
-```
+1. [Select the python interpreter](https://code.visualstudio.com/docs/python/environments#_working-with-python-interpreters) for the MODFLOW 6 workspace. To select the python interpreter run:
 
-Setting the formatter up in this way will cause a source file to reformat with each explicit save.
+    * Press `Ctrl + Shift + P` in VSCode.
+    * Type `Python: Select Interpreter`.
+    * Select `Run Task` (press Enter).
+    * Select the `modflow6` conda environment.
 
-### Running VSCode
+2. Open a new terminal in VSCode and determine the path to the Fortran language server (`fortls`) and `fprettify` using the syntax for your terminal (`cmd`, `bash`, `powershell`).
 
-Open the top level `modflow6` repository directory on your system when starting VSCode. The program will
-then look for the `modflow6/.vscode` directory to discover settings relevant to your session.
+3. Set the setting "fortran.fortls.path" and "fortran.formatting.path":
 
-A nice alternative on any system is to start VSCode from the shell. For example, in a bash or git bash
-shell (windows), change to the `modflow6` directory and execute the command:
+    ```json
+    {
+        "fortran.fortls.path": "/path/to/fortls",
+        "fortran.formatting.path": "/path/to/fprettify",
+    }
+    ```
+
+4. The fortran formatter can be integrated with VSCode using the following settings:
+
+    ```json
+    {
+        "[fortran]": {
+            "editor.formatOnSave": true,
+        },
+        "fortran.formatting.formatter": "fprettify",
+        "fortran.formatting.fprettifyArgs": ["-c", "/path/to/modflow6/.fprettify.yaml"],
+    }
+    ```
+    
+    Setting the formatter up in this way will cause a source file to reformat with each explicit save.
+
+In general, to determine the path to python, the fortran language server (`fortls`), and `fprettify` in your environment run:
+
+- bash, zsh: `which <toolname>`, e.g. `which fortls`
+- cmd: `where <toolname>`, e.g. `where python`
+- PowerShell: `Get-Command <toolname>` e.g. `Get-Command fprettify`
 
-```bash
-code .
-```
+## MODFLOW 6 VSCode Tasks
 
-Note the dot. Starting in this way, VSCode will open as desired, inheriting and discovering
-expected runtime settings in the correct directory location.
+A number of MODFLOW 6 development tasks in the [`.vscode/settings.json`](./settings.json) file in the MODFLOW 6 repository. Debugging tasks are included in the `.vscode/launch.json` file, that you will create (instructions below). The tasks includes tasks to compile, debug, and format files. A number of miscellaneous tasks are also included to help prepare required files for Pull Requests (for example, makefiles).
 
 ### Compiling
 
 In order to compile Fortran source run:
 
 * Press `Ctrl + Shift + P` in VSCode.
-* Type `Tasks`.
-* Select `Run Task` (press Enter).
+* Type `Tasks: Run Build Task` (press Enter).
 * Select the suitable task for your situation.
 
+Tasks are available to build MODFLOW 6 using gfortran and Intel Fortran (ifort). If you have used one of the Rebuild tasks, Build tasks for the same compiler can be used to just recompile modified source files and relink the object files. There are also pixi tasks for MODFLOW 6 build using gfortran.
+
+Available compiling tasks include:
+
+* `Rebuild mf6 (gfortran, release)`
+* `Rebuild mf6 (ifort, release)`
+* `Rebuild mf6 (gfortran, debug)`
+* `Rebuild mf6 (ifort, debug)`
+* `Build mf6 (gfortran, release)`
+* `Build mf6 (ifort, release)`
+* `Build mf6 (gfortran, debug)`
+* `Build mf6 (ifort, debug)`
+* `Pixi - Rebuild mf6 (gfortran, release)`
+* `Pixi - Rebuild mf6 (gfortran, debug)`
+* `Pixi - Build mf6 (gfortran, release)`
+* `Pixi - Build mf6 (gfortran, debug)`
+
+
 ### Debugging
 
 Add a `launch.json` in `.vscode` similar to this.
@@ -154,3 +183,39 @@ After building modflow, you can start debugging.
 
 - Set a breakpoint somewhere in the source code.
 - Press `Ctrl + F5` to start debugging.
+
+### Formatting
+
+There are a number of VSCode tasks to determine if any of the Fortran source files, python code or scripts, LaTeX files, and markdown files in the MODFLOW 6 repository need to be reformatted. 
+
+In order format file run:
+
+* Press `Ctrl + Shift + P` in VSCode.
+* Type `Tasks: Run Build Task` (press Enter).
+* Select the suitable task for your situation.
+
+Currently available formatting tasks include:
+
+* `Check spelling` - check spelling in Fortran source files, python code or scripts, LaTeX files, and markdown files
+* `Check Fortran format` - check the format of Fortran source files using `fprettify`
+* `Check python format` - check the python formatting using `ruff`
+* `Check python lint` - check python files for programmatic and stylistic errors using `ruff`
+
+The Fortran format task is only needed if the fortran formatter has not been [integrated with VSCode](#settings).
+
+### Additional tasks
+
+A number of additional VSCode tasks are available to perform common tasks that are required before making a Pull Request (see the [Developer documents](../DEVELOPER.md)). 
+
+In order run additional tasks run:
+
+* Press `Ctrl + Shift + P` in VSCode.
+* Type `Tasks: Run Build Task` (press Enter).
+* Select the suitable task for your situation.
+
+Currently available additional tasks include:
+
+* `Run update_flopy.py` - updates FloPy MODFLOW 6 classes in the modflow6 Conda environment using current MODFLOW 6 definition files.
+* `Run dfn2f90.py` - updates Fortran definitions using current MODFLOW 6 definition files.
+* `Run mf6ivar.py` - Regenerates LaTeX file for the input output guide current MODFLOW 6 definition files.
+* `Rebuild makefiles` - Rebuilds the GNU makefile for MODFLOW 6 (`mf6`), ZONEBUDGET 6 (`zbud6`), and the MODFLOW Converter (`mf5to6`).

.vscode/tasks.json

@@ -563,5 +563,62 @@
                 "clear": true
             }
         },
+        {
+            "label": "Check spelling",
+            "type": "shell",
+            "command": "python .github/common/check_spelling.py",
+            "options": {"cwd": "${workspaceFolder}"},
+            "group": {
+                "kind": "build",
+                "isDefault": true
+            },
+            "presentation": {
+                "clear": true
+            },
+            "problemMatcher": []
+        },
+        {
+            "label": "Check Fortran format",
+            "type": "shell",
+            "command": "python .github/common/check_format.py",
+            "options": {"cwd": "${workspaceFolder}"},
+            "group": {
+                "kind": "build",
+                "isDefault": true
+            },
+            "presentation": {
+                "clear": true
+            },
+            "problemMatcher": []
+        },
+        {
+            "label": "Check python format",
+            "type": "shell",
+            "command": "ruff format --config .github/common/ruff.toml --check .",
+            "options": {"cwd": "${workspaceFolder}"},
+            "group": {
+                "kind": "build",
+                "isDefault": true
+            },
+            "presentation": {
+                "clear": true
+            },
+            "problemMatcher": []
+        },
+        {
+            "label": "Check python lint",
+            "type": "shell",
+            "command": "ruff check --config .github/common/ruff.toml .",
+            "options": {"cwd": "${workspaceFolder}"},
+            "group": {
+                "kind": "build",
+                "isDefault": true
+            },
+            "presentation": {
+                "clear": true
+            },
+            "problemMatcher": []
+        },
+
     ]
 }

CONTRIBUTING.md

@@ -7,8 +7,8 @@ Contributions to MODFLOW 6 are welcome. We ask that contributors follow some gui
 
 - [Conduct](#conduct)
 - [Submissions](#submissions)
-  - [Questions](#questions)
   - [Bugs](#bugs)
+  - [Questions](#questions)
   - [Requests](#requests)
 - [Source code](#source-code)
   - [Format](#format)
@@ -66,17 +66,24 @@ To submit a pull request (PR):
 
 1. To avoid duplicating effort, [search](https://github.com/MODFLOW-USGS/modflow6/pulls) for an open or closed PR that relates to your submission.
 2. Fork the MODFLOW-USGS/modflow6 repo and make your changes in a new branch, following our style and commit message guidelines and [including appropriate test cases](./DEVELOPER.md#writing-tests).
-3. [Rebuild makefiles](./DEVELOPER.md#generating-makefiles) and update MSVS project files if you added, removed, or renamed any source files.
-4. [Run the full test suite](./DEVELOPER.md#running-tests) and make sure all tests pass.
-5. Push your branch to GitHub and create a pull request to the `develop` branch.
-6. If we suggest changes:
+3. [Check the spelling and formatting](./DEVELOPER.md#formatting) of any modified or new Fortran source files, python files definition files, markdown, and LaTeX files.
+4. [Rebuild makefiles](./DEVELOPER.md#generating-makefiles) and update MSVS project files if you added, removed, or renamed any source files.
+5. [Run the full test suite](./DEVELOPER.md#running-tests) and make sure all tests pass.
+6. Push your branch to GitHub and create a pull request to the `develop` branch.
+7. If we suggest changes:
   a. make the required updates
   b. make sure tests still pass
   c. rebase your branch if needed
   d. (force) push to update your PR
 
 That's it! Thank you for your contribution!
 
+If you have installed the pixi environment you can complete steps 3 and 4 using:
+
+```shell
+pixi run prepare-pull-request
+```
+
 ## Source code
 
 ### Format