Skip to content

IFC-1811: Replace toml package with tomllib and tomli optionally #551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Parent: Merge develop into infrahub-develop
merged 6 commits into from
Sep 17, 2025
Merged

IFC-1811: Replace toml package with tomllib and tomli optionally #551

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f942609
Replace toml package with tomllib and tomli optionally
solababs Sep 17, 2025
b792772
fix for mypy errors
solababs Sep 17, 2025
db2e183
update config, add pre-commit config
solababs Sep 17, 2025
2ea0aeb
Make tomli package optional
solababs Sep 17, 2025
4ebe000
Add towncrier housekeeping message
solababs Sep 17, 2025
1b7e6a7
update python to uppercase p
solababs Sep 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ jobs:
- name: "Install dependencies"
run: npm install
- name: "Setup Python environment"
run: "pip install invoke toml"
run: "pip install invoke"
- name: "Build docs website"
run: "invoke docs"

Expand Down Expand Up @@ -176,7 +176,7 @@ jobs:
- name: "Install dependencies"
run: "poetry install --no-interaction --no-ansi --extras ctl"
- name: "Setup environment"
run: "pip install invoke toml"
run: "poetry run pip install invoke"
- name: "Validate generated documentation"
run: "poetry run invoke docs-validate"

Expand Down Expand Up @@ -236,7 +236,11 @@ jobs:
run: |
pipx install poetry==${{ needs.prepare-environment.outputs.POETRY_VERSION }} --python python${{ matrix.python-version }}
poetry config virtualenvs.create true --local
pip install invoke toml codecov
pip install invoke codecov
- name: "Install tomli for Python < 3.11"
if: matrix.python-version == '3.9' || matrix.python-version == '3.10'
run: |
pip install tomli
- name: "Install Package"
run: "poetry install --all-extras"
- name: "Mypy Tests"
Expand Down Expand Up @@ -289,7 +293,7 @@ jobs:
run: |
pipx install poetry==${{ needs.prepare-environment.outputs.POETRY_VERSION }}
poetry config virtualenvs.create true --local
pip install invoke toml codecov
pip install invoke codecov
- name: "Install Package"
run: "poetry install --all-extras"
- name: "Integration Tests"
Expand Down Expand Up @@ -362,7 +366,7 @@ jobs:
# run: |
# pipx install poetry==${{ needs.prepare-environment.outputs.POETRY_VERSION }}
# poetry config virtualenvs.create true --local
# pip install invoke toml codecov
# pip install invoke codecov

# - name: "Install Package"
# run: "poetry install --all-extras"
Expand Down
22 changes: 22 additions & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: trailing-whitespace
- id: check-ast
- id: check-case-conflict
- id: check-merge-conflict
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer

- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.11.9
hooks:
# Run the linter.
- id: ruff
args: [--fix]
# Run the formatter.
- id: ruff-format
2 changes: 1 addition & 1 deletion docs/docusaurus.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ const config: Config = {
additionalLanguages: ["bash", "python", "markup-templating", "django", "json", "toml", "yaml"],
},
} satisfies Preset.ThemeConfig,

markdown: {
format: "mdx",
preprocessor: ({ filePath, fileContent }) => {
Expand Down
9 changes: 7 additions & 2 deletions infrahub_sdk/ctl/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,18 @@

from __future__ import annotations

import sys
from pathlib import Path

import toml
import typer
from pydantic import Field, ValidationError, field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict

if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib

DEFAULT_CONFIG_FILE = "infrahubctl.toml"
ENVVAR_CONFIG_FILE = "INFRAHUBCTL_CONFIG"
INFRAHUB_REPO_CONFIG_FILE = ".infrahub.yml"
Expand Down Expand Up @@ -59,7 +64,7 @@ def load(self, config_file: str | Path = "infrahubctl.toml", config_data: dict |

if config_file.is_file():
config_string = config_file.read_text(encoding="utf-8")
config_tmp = toml.loads(config_string)
config_tmp = tomllib.loads(config_string)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Handle TOML parse errors and fix doc typos.

Currently TOML decode errors bubble unhandled; load_and_exit only catches ValidationError. Catch tomllib.TOMLDecodeError (works for both tomllib and tomli alias) and show a clear message. Also fix typos and align parameter names; prefer DEFAULT_CONFIG_FILE constant in signatures.

Apply:

-    def load(self, config_file: str | Path = "infrahubctl.toml", config_data: dict | None = None) -> None:
+    def load(self, config_file: str | Path = DEFAULT_CONFIG_FILE, config_data: dict | None = None) -> None:
@@
-        if config_file.is_file():
-            config_string = config_file.read_text(encoding="utf-8")
-            config_tmp = tomllib.loads(config_string)
+        if config_file.is_file():
+            config_string = config_file.read_text(encoding="utf-8")
+            try:
+                config_tmp = tomllib.loads(config_string)
+            except tomllib.TOMLDecodeError as exc:
+                print(f"Configuration file parse error: {config_file} — {exc}")  # consider typer.secho
+                raise typer.Abort()
@@
-    def load_and_exit(self, config_file: str | Path = "infrahubctl.toml", config_data: dict | None = None) -> None:
+    def load_and_exit(self, config_file: str | Path = DEFAULT_CONFIG_FILE, config_data: dict | None = None) -> None:
@@
-        """Calls load, but wraps it in a try except block.
+        """Calls load, but wraps it in a try/except block.
@@
-        This is done to handle a ValidationErorr which is raised when settings are specified but invalid.
+        This handles a ValidationError when settings are specified but invalid.
@@
-            config_file_name (str, optional): [description]. Defaults to "pyprojectctl.toml".
+            config_file (str | Path, optional): Path to TOML config. Defaults to "infrahubctl.toml".
@@
-        except ValidationError as exc:
+        except ValidationError as exc:
             print(f"Configuration not valid, found {len(exc.errors())} error(s)")
             for error in exc.errors():
                 loc_str = [str(item) for item in error["loc"]]
                 print(f"  {'/'.join(loc_str)} | {error['msg']} ({error['type']})")
             raise typer.Abort()

Optional: replace print with typer.echo/secho for CLI UX.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
config_tmp = tomllib.loads(config_string)
try:
config_tmp = tomllib.loads(config_string)
except tomllib.TOMLDecodeError as exc:
print(f"Configuration file parse error: {config_file} — {exc}") # consider typer.secho
raise typer.Abort()
🤖 Prompt for AI Agents 
In infrahub_sdk/ctl/config.py around line 66, the call to tomllib.loads can
raise tomllib.TOMLDecodeError which is currently unhandled; update load_and_exit
to catch tomllib.TOMLDecodeError (compatible with tomli alias), log or echo a
clear, user-facing message about invalid TOML (include the error message), and
exit similarly to ValidationError handling; also fix docstring/print typos and
align function parameter names to use the DEFAULT_CONFIG_FILE constant in
function signatures and docs, and optionally replace print calls with
typer.echo/secho for better CLI output.


self._settings = Settings(**config_tmp)
return
Expand Down
Loading