Skip to content

Commit 5cb8dc5

Browse files
peytondmurraypeytondmurray
authored
Merge pull request #52 from andrewfulton9/linting
Adds CI Linting
2 parents 8f0e16d + 6640b24 commit 5cb8dc5

File tree

13 files changed

+282
-149
lines changed

13 files changed

+282
-149
lines changed

‎.github/workflows/ci-lint.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
name: pre-commit
2+
3+
on:
4+
pull_request:
5+
push:
6+
branches: [master]
7+
8+
jobs:
9+
pre-commit:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/[email protected]
13+
with:
14+
# Ensure the full history is fetched
15+
# This is required to run pre-commit on a specific set of commits
16+
# TODO: Remove this when all the pre-commit issues are fixed
17+
fetch-depth: 0
18+
- uses: actions/[email protected]
19+
with:
20+
python-version: 3.13
21+
- uses: pre-commit/[email protected]

‎.pre-commit-config.yaml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# pre-commit is a tool to perform a predefined set of tasks manually and/or
2+
# automatically before git commits are made.
3+
#
4+
# Config reference: https://pre-commit.com/#pre-commit-configyaml---top-level
5+
#
6+
# Common tasks
7+
#
8+
# - Register git hooks: pre-commit install --install-hooks
9+
# - Run on all files: pre-commit run --all-files
10+
#
11+
# These pre-commit hooks are run as CI.
12+
#
13+
# NOTE: if it can be avoided, add configs/args in pyproject.toml or below instead of creating a new `.config.file`.
14+
# https://pre-commit.ci/#configuration
15+
ci:
16+
autoupdate_schedule: monthly
17+
autofix_commit_msg: |
18+
[pre-commit.ci] Apply automatic pre-commit fixes
19+
20+
repos:
21+
# general
22+
- repo: https://github.com/pre-commit/pre-commit-hooks
23+
rev: v4.6.0
24+
hooks:
25+
- id: end-of-file-fixer
26+
exclude: '\.svg$'
27+
- id: trailing-whitespace
28+
exclude: '\.svg$'
29+
- id: check-json
30+
- id: check-yaml
31+
args: [--allow-multiple-documents, --unsafe]
32+
- id: check-toml
33+
34+
- repo: https://github.com/astral-sh/ruff-pre-commit
35+
rev: v0.5.6
36+
hooks:
37+
- id: ruff
38+
args: ["--fix"]
39+
- id: ruff-format

‎RELEASE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@
112112
* Bump the Ubuntu version on which TFMD is tested to 20.04 (previously
113113
was 16.04).
114114
* Bumped the minimum bazel version required to build `tfmd` to 6.1.0.
115-
* Depends on `protobuf>=4.25.2,<5` for Python 3.11 and on
115+
* Depends on `protobuf>=4.25.2,<5` for Python 3.11 and on
116116
`protobuf>3.20.3,<4.21` for 3.9 and 3.10.
117117
* Depends on `googleapis-common-protos>=1.56.4,<2` for Python 3.11 and on
118118
`googleapis-common-protos>=1.52.0,<2` for 3.9 and 3.10.

‎ruff.toml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
line-length = 88
2+
3+
[lint]
4+
select = [
5+
# pycodestyle
6+
"E",
7+
"W",
8+
# Pyflakes
9+
"F",
10+
# pyupgrade
11+
"UP",
12+
# flake8-bugbear
13+
"B",
14+
# flake8-simplify
15+
"SIM",
16+
# isort
17+
"I",
18+
# pep8 naming
19+
"N",
20+
# pydocstyle
21+
"D",
22+
# annotations
23+
"ANN",
24+
# debugger
25+
"T10",
26+
# flake8-pytest
27+
"PT",
28+
# flake8-return
29+
"RET",
30+
# flake8-unused-arguments
31+
"ARG",
32+
# flake8-fixme
33+
"FIX",
34+
# flake8-eradicate
35+
"ERA",
36+
# pandas-vet
37+
"PD",
38+
# numpy-specific rules
39+
"NPY",
40+
]
41+
42+
ignore = [
43+
"D104", # Missing docstring in public package
44+
"D100", # Missing docstring in public module
45+
"D211", # No blank line before class
46+
"PD901", # Avoid using 'df' for pandas dataframes. Perfectly fine in functions with limited scope
47+
"ANN201", # Missing return type annotation for public function (makes no sense for NoneType return types...)
48+
"ANN101", # Missing type annotation for `self`
49+
"ANN204", # Missing return type annotation for special method
50+
"ANN002", # Missing type annotation for `*args`
51+
"ANN003", # Missing type annotation for `**kwargs`
52+
"D105", # Missing docstring in magic method
53+
"D203", # 1 blank line before after class docstring
54+
"D204", # 1 blank line required after class docstring
55+
"D413", # 1 blank line after parameters
56+
"SIM108", # Simplify if/else to one line; not always clearer
57+
"D206", # Docstrings should be indented with spaces; unnecessary when running ruff-format
58+
"E501", # Line length too long; unnecessary when running ruff-format
59+
"W191", # Indentation contains tabs; unnecessary when running ruff-format
60+
61+
# REMOVE AFTER FIXING
62+
"ANN001", # Missing type annotation for function argument `args`
63+
"ANN202", # Missing Missing return type annotation for private function
64+
"D103", # Missing docstring in public function
65+
"D101", # Missing docstring in public class
66+
]
67+
68+
69+
[lint.per-file-ignores]
70+
"__init__.py" = ["F401"]

‎setup.py‎

Lines changed: 94 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -18,130 +18,140 @@
1818
import shutil
1919
import subprocess
2020

21-
import setuptools
22-
from setuptools import find_packages
23-
from setuptools import setup
2421
# pylint: disable=g-bad-import-order
2522
# It is recommended to import setuptools prior to importing distutils to avoid
2623
# using legacy behavior from distutils.
2724
# https://setuptools.readthedocs.io/en/latest/history.html#v48-0-0
2825
from distutils.command import build
26+
27+
import setuptools
28+
from setuptools import find_packages, setup
29+
2930
# pylint: enable=g-bad-import-order
3031

3132

3233
class _BuildCommand(build.build):
33-
"""Build everything that is needed to install.
34+
"""Build everything that is needed to install.
3435
35-
This overrides the original distutils "build" command to to run bazel_build
36-
command before any sub_commands.
36+
This overrides the original distutils "build" command to to run bazel_build
37+
command before any sub_commands.
3738
38-
build command is also invoked from bdist_wheel and install command, therefore
39-
this implementation covers the following commands:
40-
- pip install . (which invokes bdist_wheel)
41-
- python setup.py install (which invokes install command)
42-
- python setup.py bdist_wheel (which invokes bdist_wheel command)
43-
"""
39+
build command is also invoked from bdist_wheel and install command, therefore
40+
this implementation covers the following commands:
41+
- pip install . (which invokes bdist_wheel)
42+
- python setup.py install (which invokes install command)
43+
- python setup.py bdist_wheel (which invokes bdist_wheel command)
44+
"""
4445

45-
def _build_cc_extensions(self):
46-
return True
46+
def _build_cc_extensions(self):
47+
return True
4748

48-
# Add "bazel_build" command as the first sub_command of "build". Each
49-
# sub_command of "build" (e.g. "build_py", "build_ext", etc.) is executed
50-
# sequentially when running a "build" command, if the second item in the tuple
51-
# (predicate method) is evaluated to true.
52-
sub_commands = [
53-
('bazel_build', _build_cc_extensions),
54-
] + build.build.sub_commands
49+
# Add "bazel_build" command as the first sub_command of "build". Each
50+
# sub_command of "build" (e.g. "build_py", "build_ext", etc.) is executed
51+
# sequentially when running a "build" command, if the second item in the tuple
52+
# (predicate method) is evaluated to true.
53+
sub_commands = [
54+
("bazel_build", _build_cc_extensions),
55+
] + build.build.sub_commands
5556

5657

5758
class _BazelBuildCommand(setuptools.Command):
58-
"""Build Bazel artifacts and move generated files to the ."""
59-
60-
def initialize_options(self):
61-
pass
62-
63-
def finalize_options(self):
64-
self._bazel_cmd = shutil.which('bazel')
65-
if not self._bazel_cmd:
66-
raise RuntimeError(
67-
'Could not find "bazel" binary. Please visit '
68-
'https://docs.bazel.build/versions/master/install.html for '
69-
'installation instruction.')
70-
if platform.system() == 'Windows':
71-
self._additional_build_options = ['--copt=-DWIN32_LEAN_AND_MEAN']
72-
else:
73-
self._additional_build_options = []
74-
75-
def run(self):
76-
subprocess.check_call(
77-
[self._bazel_cmd, 'run',
78-
'--compilation_mode', 'opt',
79-
*self._additional_build_options,
80-
'//tensorflow_metadata:move_generated_files'],
81-
# Bazel should be invoked in a directory containing bazel WORKSPACE
82-
# file, which is the root directory.
83-
cwd=os.path.dirname(os.path.realpath(__file__)),)
84-
85-
86-
with open('tensorflow_metadata/version.py') as fp:
87-
globals_dict = {}
88-
exec(fp.read(), globals_dict) # pylint: disable=exec-used
59+
"""Build Bazel artifacts and move generated files to the ."""
60+
61+
def initialize_options(self):
62+
pass
63+
64+
def finalize_options(self):
65+
self._bazel_cmd = shutil.which("bazel")
66+
if not self._bazel_cmd:
67+
raise RuntimeError(
68+
'Could not find "bazel" binary. Please visit '
69+
"https://docs.bazel.build/versions/master/install.html for "
70+
"installation instruction."
71+
)
72+
if platform.system() == "Windows":
73+
self._additional_build_options = ["--copt=-DWIN32_LEAN_AND_MEAN"]
74+
else:
75+
self._additional_build_options = []
76+
77+
def run(self):
78+
subprocess.check_call(
79+
[
80+
self._bazel_cmd,
81+
"run",
82+
"--compilation_mode",
83+
"opt",
84+
*self._additional_build_options,
85+
"//tensorflow_metadata:move_generated_files",
86+
],
87+
# Bazel should be invoked in a directory containing bazel WORKSPACE
88+
# file, which is the root directory.
89+
cwd=os.path.dirname(os.path.realpath(__file__)),
90+
)
91+
92+
93+
with open("tensorflow_metadata/version.py") as fp:
94+
globals_dict = {}
95+
exec(fp.read(), globals_dict) # pylint: disable=exec-used
8996

9097
# tf.Metadata version.
91-
__version__ = globals_dict['__version__']
98+
__version__ = globals_dict["__version__"]
9299

93100

94101
# Note: In order for the README to be rendered correctly, make sure to have the
95102
# following minimum required versions of the respective packages when building
96103
# and uploading the zip/wheel package to PyPI:
97104
# setuptools >= 38.6.0, wheel >= 0.31.0, twine >= 1.11.0
98105
# Get the long description from the README file.
99-
with open('README.md') as fp:
100-
_LONG_DESCRIPTION = fp.read()
106+
with open("README.md") as fp:
107+
_LONG_DESCRIPTION = fp.read()
101108

102109
setup(
103-
name='tensorflow-metadata',
110+
name="tensorflow-metadata",
104111
version=__version__,
105-
author='Google Inc.',
106-
author_email='[email protected]',
107-
license='Apache 2.0',
112+
author="Google Inc.",
113+
author_email="[email protected]",
114+
license="Apache 2.0",
108115
classifiers=[
109-
'Development Status :: 5 - Production/Stable',
110-
'Intended Audience :: Developers',
111-
'Intended Audience :: Education',
112-
'Intended Audience :: Science/Research',
113-
'License :: OSI Approved :: Apache Software License',
114-
'Operating System :: OS Independent',
115-
'Programming Language :: Python',
116-
'Programming Language :: Python :: 3',
117-
'Programming Language :: Python :: 3.9',
118-
'Programming Language :: Python :: 3.10',
119-
'Programming Language :: Python :: 3.11',
120-
'Programming Language :: Python :: 3 :: Only',
121-
'Topic :: Scientific/Engineering :: Artificial Intelligence',
122-
'Topic :: Scientific/Engineering :: Mathematics',
123-
'Topic :: Software Development',
124-
'Topic :: Software Development :: Libraries',
125-
'Topic :: Software Development :: Libraries :: Python Modules',
116+
"Development Status :: 5 - Production/Stable",
117+
"Intended Audience :: Developers",
118+
"Intended Audience :: Education",
119+
"Intended Audience :: Science/Research",
120+
"License :: OSI Approved :: Apache Software License",
121+
"Operating System :: OS Independent",
122+
"Programming Language :: Python",
123+
"Programming Language :: Python :: 3",
124+
"Programming Language :: Python :: 3.9",
125+
"Programming Language :: Python :: 3.10",
126+
"Programming Language :: Python :: 3.11",
127+
"Programming Language :: Python :: 3 :: Only",
128+
"Topic :: Scientific/Engineering :: Artificial Intelligence",
129+
"Topic :: Scientific/Engineering :: Mathematics",
130+
"Topic :: Software Development",
131+
"Topic :: Software Development :: Libraries",
132+
"Topic :: Software Development :: Libraries :: Python Modules",
126133
],
127134
namespace_packages=[],
128135
install_requires=[
129-
'absl-py>=0.9,<3.0.0',
136+
"absl-py>=0.9,<3.0.0",
130137
'googleapis-common-protos>=1.56.4,<2;python_version>="3.11"',
131138
'protobuf>=4.25.2;python_version>="3.11"',
132139
'protobuf>=4.21.6,<4.22;python_version<"3.11"',
133140
],
134-
python_requires='>=3.9,<4',
141+
python_requires=">=3.9,<4",
135142
packages=find_packages(),
143+
extra_requires={
144+
"dev": ["precommit"],
145+
},
136146
include_package_data=True,
137-
description='Library and standards for schema and statistics.',
147+
description="Library and standards for schema and statistics.",
138148
long_description=_LONG_DESCRIPTION,
139-
long_description_content_type='text/markdown',
140-
keywords='tensorflow metadata tfx',
141-
download_url='https://github.com/tensorflow/metadata/tags',
149+
long_description_content_type="text/markdown",
150+
keywords="tensorflow metadata tfx",
151+
download_url="https://github.com/tensorflow/metadata/tags",
142152
requires=[],
143153
cmdclass={
144-
'build': _BuildCommand,
145-
'bazel_build': _BazelBuildCommand,
154+
"build": _BuildCommand,
155+
"bazel_build": _BazelBuildCommand,
146156
},
147157
)

‎tensorflow_metadata/proto/__init__.py‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,12 @@
1313
# limitations under the License.
1414
"""Init module for tf.Metadata protos."""
1515

16-
from tensorflow_metadata.proto.v0 import anomalies_pb2
17-
from tensorflow_metadata.proto.v0 import derived_feature_pb2
18-
from tensorflow_metadata.proto.v0 import metric_pb2
19-
from tensorflow_metadata.proto.v0 import path_pb2
20-
from tensorflow_metadata.proto.v0 import problem_statement_pb2
21-
from tensorflow_metadata.proto.v0 import schema_pb2
22-
from tensorflow_metadata.proto.v0 import statistics_pb2
16+
from tensorflow_metadata.proto.v0 import (
17+
anomalies_pb2,
18+
derived_feature_pb2,
19+
metric_pb2,
20+
path_pb2,
21+
problem_statement_pb2,
22+
schema_pb2,
23+
statistics_pb2,
24+
)

‎tensorflow_metadata/proto/v0/BUILD‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,4 +66,3 @@ py_proto_library(
6666
":metadata_v0_proto",
6767
],
6868
)
69-

0 commit comments

Comments
 (0)