feat: add python utils and integrate into workflow#2176
Merged
Conversation
1a12a6d to
244dbff
Compare
pjenvey
reviewed
Mar 30, 2026
Member
pjenvey
left a comment
There was a problem hiding this comment.
I'd argue it's too strict to enforce pydocstyle on all test files, as most of them are superfluous, just repeating the test name itself. It should be easy enough to exclude them e.g. PyCQA/pydocstyle#553
| run: poetry run ruff check tools | ||
|
|
||
| - name: Python Docstring Check | ||
| run: poetry run pydocstyle -es --count --config=pyproject.toml tools |
Member
There was a problem hiding this comment.
Apparently ruff might be able to replace pydocstyle now? It has a plugin, anyway https://docs.astral.sh/ruff/settings/#lint_pydoclint_ignore-one-line-docstrings
pjenvey
approved these changes
Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a number of important Python utilities to improve the developer experience. These increase reliability of code though regular static security checks using Bandit as well as enforcing type safety through MyPy. Additinoally, some useful linting and documentation features are added though pydocstyle and isort.
Adds Makefile targets to easily run these in a local environment (using Poetry) and integrates checks into our Python actions.
All of our integration and e2e testing is in Python, as well as our management utility scripts for Postgres and Spanner, so this updates all modules to be compliant.
Addresses lingering dependency updates in one swoop, too.
Bandit Security Linting
Code Fixes
B602 HIGH —
subprocesswithshell=Trueremovedtools/integration_tests/conftest.py:60target_binaryis resolved from hardcoded paths viaos.path.exists()— never user-controlled.Replaced
Popen(target_binary, shell=True)withPopen([target_binary]).B108 MEDIUM — hardcoded
/tmp/replaced withtempfile.mkstemp()tools/tokenserver/test_scripts.py:152,167Two test temp files now use
fd, filename = tempfile.mkstemp(); os.close(fd).# nosecSuppressions (false positives)postgres/purge_ttl.py:97,110:paramspanner/purge_ttl.py:138,154spanner/count_expired_rows.py:58tableiterates over hardcoded list["batches", "bsos"]tokenserver/database.py:645syncstorage-loadtest/storage/auth.py:81,108Global Skips —
pyproject.toml:59–67"SECRET") — never production code"Ted_Koppel_is_a_robot","secret-a") — never production codeshell=Trueon a trusted internal binary path — same category as already-skipped B404Testing
New workflow steps in Python checks run, but you can check locally using the new Makefile targets.
Issue(s)
Closes STOR-519.