Some documentation stuff

Author: emersodb

.github/pull_request_template.md

@@ -1,8 +1,12 @@
 # PR Type
-[Feature | Fix | Documentation | Other() ]
+[Feature | Fix | Documentation | Other ]
 
 # Short Description
-...
+
+Clickup Ticket(s): Link(s) if applicable.
+
+Add a short description of what is in this PR.
 
 # Tests Added
-...
+
+Describe the tests that have been added to ensure the codes correctness, if applicable.

.pre-commit-config.yaml

@@ -27,12 +27,18 @@ repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: 'v0.11.13'
     hooks:
-    - id: ruff
+    - id: ruff-check
       args: [--fix, --exit-non-zero-on-fix]
       types_or: [python, jupyter]
     - id: ruff-format
       types_or: [python, jupyter]
 
+  - repo: https://github.com/jsh9/pydoclint
+    rev: '0.6.6'
+    hooks:
+      - id: pydoclint
+        args: [--style=google, --check-return-types=True, --exclude=tests]
+
   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: v1.16.0
     hooks:
@@ -71,6 +77,14 @@ repos:
       pass_filenames: false
       always_run: true
 
+  - repo: local
+    hooks:
+      - id: mypy legacy type check
+        name: mypy legacy type check
+        entry: python mypy_disallow_legacy_types.py
+        language: python
+        pass_filenames: true
+
 ci:
     autofix_commit_msg: |
         [pre-commit.ci] Add auto fixes from pre-commit.com hooks

CONTRIBUTING.md

@@ -1,26 +1,86 @@
-# Contributing to aieng-template
+# Contributing to MIDST Toolkit
 
-Thanks for your interest in contributing to the aieng-template!
+Thanks for your interest in contributing to the MIDST Toolkit!
 
 To submit PRs, please fill out the PR template along with the PR. If the PR
 fixes an issue, don't forget to link the PR to the issue!
 
 ## Pre-commit hooks
 
-Once the python virtual environment is setup, you can run pre-commit hooks using:
+```bash
+pre-commit install
+```
 
+To run the checks, some of which will automatically re-format your code to fit the standards, you can run
 ```bash
 pre-commit run --all-files
 ```
+It can also be run on a subset of files by omitting the `--all-files` option and pointing to specific files or folders.
+
+If you're using VS Code for development, pre-commit should setup git hooks that execute the pre-commit checks each
+time you check code into your branch through the integrated source-control as well. This will ensure that each of your
+commits conform to the desired format before they are run remotely and without needing to remember to run the checks
+before pushing to a remote. If this isn't done automatically, you can find instructions for setting up these hooks
+manually online.
 
 ## Coding guidelines
 
 For code style, we recommend the [PEP 8 style guide](https://peps.python.org/pep-0008/).
 
-For docstrings we use [numpy format](https://numpydoc.readthedocs.io/en/latest/format.html).
+For code documentation, we try to adhere to the Google docstring style
+(See [here](https://google.github.io/styleguide/pyguide.html), Section: Comments and Doc-strings). The implementation
+of an extensive set of comments for the code in this repository is a work-in-progress. However, we are continuing to
+work towards a better commented state for the code. For development, as stated in the style guide,
+__any non-trivial or non-obvious methods added to the library should have a doc string__. For our library this
+applies only to code added to the main library in `midst_toolkit`. Examples, research code, and tests need not
+incorporate  the strict rules of documentation, though clarifying and helpful comments in that code is also
+__strongly encouraged__.
+
+> [!NOTE]
+> As a matter of convention choice, classes are documented through their `__init__` functions rather than at the
+> "class" level.
+
+If you are using VS Code a very helpful integration is available to facilitate the creation of properly formatted
+doc-strings called autoDocstring [VS Code Page](https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring)
+and [Documentation](https://github.com/NilsJPWerner/autoDocstring). This tool will automatically generate a docstring
+template when starting a docstring with triple quotation marks (`"""`). To get the correct format, the following
+settings should be prescribed in your VS Code settings JSON:
+
+```json
+{
+    "autoDocstring.customTemplatePath": "",
+    "autoDocstring.docstringFormat": "google",
+    "autoDocstring.generateDocstringOnEnter": true,
+    "autoDocstring.guessTypes": true,
+    "autoDocstring.includeExtendedSummary": false,
+    "autoDocstring.includeName": false,
+    "autoDocstring.logLevel": "Info",
+    "autoDocstring.quoteStyle": "\"\"\"",
+    "autoDocstring.startOnNewLine": true
+}
+```
 
 We use [ruff](https://docs.astral.sh/ruff/) for code formatting and static code
-analysis. Ruff checks various rules including [flake8](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-flake8). The pre-commit hooks show errors which you need to fix before submitting a PR.
+analysis. Ruff checks various rules including
+[flake8](https://docs.astral.sh/ruff/faq/#how-does-ruff-compare-to-flake8). The pre-commit hooks show errors which
+you need to fix before submitting a PR.
 
 Last but not the least, we use type hints in our code which is then checked using
 [mypy](https://mypy.readthedocs.io/en/stable/).
+
+**Note**: We use the modern mypy types introduced in Python 3.10 and above. See some of the
+[documentation here](https://mypy.readthedocs.io/en/stable/builtin_types.html)
+
+For example, this means that we're using `list[str], tuple[int, int], tuple[int, ...], dict[str, int], type[C]` as
+built-in types and `Iterable[int], Sequence[bool], Mapping[str, int], Callable[[...], ...]` from collections.abc
+(as now recommended by mypy).
+
+We also use the new Optional and Union specification style:
+```python
+Optional[typing_stuff] -> typing_stuff | None
+Union[typing1, typing2] -> typing1 | typing2
+Optional[Union[typing1, typing2]] -> typing1 | typing2 | None
+```
+
+There is a custom script that enforces this style. It is not infallible. So if there is an issue with it please fix or
+report it to us.

README.md

@@ -1,4 +1,4 @@
-# AI Engineering template (with uv)
+# MIDST Toolkit
 
 ----------------------------------------------------------------------------------------
 
@@ -8,10 +8,9 @@
 [![codecov](https://codecov.io/github/VectorInstitute/midst-toolkit/graph/badge.svg?token=83MYFZ3UPA)](https://codecov.io/github/VectorInstitute/midst-toolkit)
 ![GitHub License](https://img.shields.io/github/license/VectorInstitute/midst-toolkit)
 
-A template repo for AI Engineering projects (using ``python``) and ``uv``. This
-template is like our original AI Engineering [template](https://github.com/VectorInstitute/aieng-template),
-however, unlike how that template uses poetry, this one uses uv for dependency
-management (as well as packaging and publishing).
+A toolkit for facilitating MIA resiliency testing on diffusion-model-based synthetic tabular data. Many of the attacks
+included in this toolkit are based on the most success ones used in the
+[2025 SaTML MIDST Competition](https://vectorinstitute.github.io/MIDST/).
 
 ## 🧑🏿‍💻 Developing
 
@@ -40,34 +39,3 @@ run:
 ```bash
 uv sync --no-group docs
 ```
-
-If you're coming from `poetry` then you'll notice that the virtual environment
-is actually stored in the project root folder and is by default named as `.venv`.
-The other important note is that while `poetry` uses a "flat" layout of the project,
-`uv` opts for the the "src" layout. (For more info, see [here](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/))
-
-### Poetry to UV
-
-The table below provides the `uv` equivalent counterparts for some of the more
-common `poetry` commands.
-
-| Poetry                                               | UV                                          |
-|------------------------------------------------------|---------------------------------------------|
-| `poetry new <project-name>`  # creates new project   | `uv init <project-name>`                    |
-| `poetry install`  # installs existing project        | `uv sync`                                   |
-| `poetry install --with docs,test`                    | `uv sync --group docs --group test`         |
-| `poetry add numpy`                                   | `uv add numpy`                              |
-| `poetry add pytest pytest-asyncio --groups dev`      | `uv add pytest pytest-asyncio --groups dev` |
-| `poetry remove numpy`                                | `uv remove numpy`                           |
-| `poetry lock`                                        | `uv lock`                                   |
-| `poetry run <cmd>`  # runs cmd with the project venv | `uv run <cmd>`                              |
-| `poetry build`                                       | `uv build`                                  |
-| `poetry publish`                                     | `uv publish`                                |
-| `poetry cache clear pypi --all`                      | `uv cache clean`                            |
-
-For the full list of `uv` commands, you can visit the official [docs](https://docs.astral.sh/uv/reference/cli/#uv).
-
-### Tidbit
-
-If you're curious about what "uv" stands for, it appears to have been more or
-less chosen [randomly](https://github.com/astral-sh/uv/issues/1349#issuecomment-1986451785).

mkdocs.yml

@@ -34,7 +34,7 @@ plugins:
       handlers:
         python:
           options:
-            docstring_style: numpy
+            docstring_style: google
             members_order: source
             separate_signature: true
             show_overloads: true

mypy_disallow_legacy_types.py

@@ -0,0 +1,68 @@
+import re
+import sys
+from collections.abc import Set
+
+
+# List of files that we want to skip with this check. Currently empty.
+files_to_ignore: Set[str] = {"Empty"}
+file_types_to_ignore: Set[str] = {".png", ".pkl", ".pt", ".md", ".svg", ".ico"}
+# List of disallowed types to search for that should no longer be imported from the typing library. These types
+# have been migrated to either collections.abc or into core python
+disallowed_types = [
+    "Union",
+    "Optional",
+    "List",
+    "Dict",
+    "Sequence",
+    "Set",
+    "Callable",
+    "Iterable",
+    "Hashable",
+    "Generator",
+    "Tuple",
+    "Mapping",
+    "Type",
+]
+
+
+type_or = "|".join(disallowed_types)
+comma_separated_types = ", ".join(disallowed_types)
+file_suffixes = "|".join(file_types_to_ignore)
+file_type_regex = rf".*({file_suffixes})$"
+
+
+def filter_files_to_ignore(file_paths: list[str]) -> list[str]:
+    file_paths = [file_path for file_path in file_paths if file_path not in files_to_ignore]
+    file_paths = [file_path for file_path in file_paths if not re.match(file_type_regex, file_path)]
+    return file_paths
+
+
+def construct_same_line_import_regex() -> str:
+    return rf"from typing import ([^\n]*?, )*({type_or})(\n|, [^\n]*?\n)"
+
+
+def construct_multi_line_import_regex() -> str:
+    return rf"from typing import \(\n(\s{{4}}.*,\n)*\s{{4}}({type_or}),\n(\s{{4}}.*,\n)*\)$"
+
+
+same_line_import_re = construct_same_line_import_regex()
+multi_line_import_re = construct_multi_line_import_regex()
+
+
+def discover_legacy_imports(file_paths: list[str]) -> None:
+    file_paths = filter_files_to_ignore(file_paths=file_paths)
+    for file_path in file_paths:
+        with open(file_path, mode="r") as file_handle:
+            file_contents = file_handle.read()
+            same_line_match = re.search(same_line_import_re, file_contents, flags=re.MULTILINE)
+            multi_line_match = re.search(multi_line_import_re, file_contents, flags=re.MULTILINE)
+            if same_line_match or multi_line_match:
+                raise ValueError(
+                    f"A legacy mypy type is being imported in file {file_path}. "
+                    f"Disallowed imports from the typing library are: {comma_separated_types}"
+                )
+
+
+if __name__ == "__main__":
+    file_relative_paths = sys.argv[1:]
+    discover_legacy_imports(file_relative_paths)

pyproject.toml

@@ -64,6 +64,7 @@ extra_checks = true
 
 [tool.ruff]
 include = ["*.py", "pyproject.toml", "*.ipynb"]
+exclude = ["mypy_disallow_legacy_types.py"]
 line-length = 119
 
 [tool.ruff.format]
@@ -97,6 +98,8 @@ ignore = [
     "E501", # line too long
     "D203", # 1 blank line required before class docstring
     "D213", # Multi-line docstring summary should start at the second line
+    "D100", # Ignore module level docstrings requirement
+    "D104", # Ignore package level docstrings requirement
     "PLR2004", # Replace magic number with named constant
     "PLR0913", # Too many arguments
     "COM812", # Missing trailing comma
@@ -111,10 +114,9 @@ ignore-names = ["X*", "setUp"]
 
 [tool.ruff.lint.isort]
 lines-after-imports = 2
-line_length = 119
 
 [tool.ruff.lint.pydocstyle]
-convention = "numpy"
+convention = "google"
 
 [tool.ruff.lint.pycodestyle]
 max-doc-length = 119

src/midst_toolkit/__init__.py

@@ -1,11 +1,7 @@
-"""Top level module."""
-
-
 def hello() -> str:
-    """UV's hello world.
+    """Hello function.
 
-    Returns
-    -------
-        str: A friendly hello.
+    Returns:
+        str: Hello world
     """
     return "Hello from midst-toolkit!"

src/midst_toolkit/bar.py

@@ -1,25 +1,10 @@
-"""bar module."""
-
-
 def bar(foo: str) -> str:
-    """Return input concatenated with 'bar'.
-
-    Parameters
-    ----------
-    foo : str
-        Input string to be concatenated with 'bar'.
-
-    Returns
-    -------
-    str
-        Concatenated string.
+    """Bar function.
 
-    Examples
-    --------
-    >>> bar("foo")
-    'barfoo'
-    >>> bar("baz")
-    'barbaz'
+    Args:
+        foo (str): Foo string
 
+    Returns:
+        str: A modified string
     """
     return f"bar{foo}"

src/midst_toolkit/foo.py

@@ -1,25 +1,10 @@
-"""foo module."""
-
-
 def foo(bar: str) -> str:
-    """Return input concatenated with 'foo'.
-
-    Parameters
-    ----------
-    bar : str
-        Input string to be concatenated with 'foo'.
-
-    Returns
-    -------
-    str
-        Concatenated string.
+    """Foo function.
 
-    Examples
-    --------
-    >>> foo("bar")
-    'foobar'
-    >>> foo("baz")
-    'foobaz'
+    Args:
+        bar (str): Bar string
 
+    Returns:
+        str: String
     """
     return f"foo{bar}"