Skip to content

ci: add api doc generation #481

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: stable
Choose a base branch
from
Open

ci: add api doc generation #481

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
d69b7f6
docs: generate API docs
a-delannoy Jul 23, 2025
36610dd
docs: fix docusaurus interpretation of mdx files with Icon in it
a-delannoy Jul 23, 2025
89511ae
docs: configure docusaurus sidebar
a-delannoy Jul 23, 2025
469bf19
chore: add invoke task to generate the documentation
a-delannoy Jul 23, 2025
9b881e8
build: add mdxify package
a-delannoy Jul 23, 2025
c016809
ci: add a rule for checking the SDK API obsolescence
a-delannoy Jul 23, 2025
1484494
style: fix some python lint error
a-delannoy Jul 23, 2025
1acb7b0
style: fix some mispelling and style issue in docstrings
a-delannoy Jul 23, 2025
ff31be5
docs: update API documentation
a-delannoy Jul 23, 2025
9d0a674
style: fix linting issues in the generated documentation
a-delannoy Jul 23, 2025
ce543b8
ci: update the CI rule to fix linting on the generated documentation
a-delannoy Jul 23, 2025
0619e37
build: move mdxify to dev group in poetry
a-delannoy Jul 23, 2025
f733340
ci: update the CI rule to install markdownlint for properly fixing ge…
a-delannoy Jul 24, 2025
d017fcf
ci: troubleshoot CI rule
a-delannoy Jul 24, 2025
0703af8
docs: update docs after rebase
a-delannoy Jul 24, 2025
9f99440
ci: resume git diff check
a-delannoy Jul 24, 2025
ed095d2
ci: exclude the newly generated documentation from vale inspection
a-delannoy Jul 25, 2025
fb08b5f
docs: update API docs after rebasing
a-delannoy Jul 25, 2025
4980873
ci: fine tune find command
a-delannoy Jul 25, 2025
e101d9a
chore: add towncrier changelog fragment for #201
a-delannoy Jul 25, 2025
9f13b94
docs: relocate the API reference on the sidebar
a-delannoy Jul 28, 2025
59a7645
refactor: update invoke task for generate API docs
a-delannoy Aug 1, 2025
cf30309
docs: update the documentation to only a specified subset
a-delannoy Aug 1, 2025
2321af2
Merge branch 'stable' into ady-add-api-doc-generation
a-delannoy Aug 1, 2025
a0a8686
docs: update documentation
a-delannoy Aug 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 47 additions & 5 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
token: ${{ github.token }}
filters: .github/file-filters.yml

# ------------------------------------------ All Linter ------------------------------------------
# ------------------------------------------ All Linter ------------------------------------------

yaml-lint:
if: needs.files-changed.outputs.yaml == 'true'
Expand Down Expand Up @@ -84,7 +84,6 @@ jobs:
- name: "Linting: ruff format"
run: "ruff format --check --diff ."


markdown-lint:
if: |
needs.files-changed.outputs.documentation == 'true' ||
Expand Down Expand Up @@ -119,7 +118,6 @@ jobs:
env:
SHELLCHECK_OPTS: --exclude=SC2086 --exclude=SC2046 --exclude=SC2004 --exclude=SC2129


documentation:
defaults:
run:
Expand All @@ -141,7 +139,7 @@ jobs:
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
cache: "npm"
cache-dependency-path: docs/package-lock.json
- name: "Install dependencies"
run: npm install
Expand Down Expand Up @@ -180,6 +178,50 @@ jobs:
- name: "Validate generated documentation"
run: "poetry run invoke docs-validate"

check-api-documentation-obsolescence:
if: |
always() && !cancelled() &&
!contains(needs.*.result, 'failure') &&
!contains(needs.*.result, 'cancelled') &&
(needs.files-changed.outputs.python == 'true') || (needs.files-changed.outputs.documentation_generated == 'true')
needs: ["prepare-environment", "files-changed", "yaml-lint", "python-lint"]
runs-on: "ubuntu-22.04"
env:
DOCS_COMMAND: "poetry run invoke generate-sdk-api-docs"
SDK_API_DOCS_DIR: "docs/docs/python-sdk/sdk_ref"
timeout-minutes: 5
steps:
- name: "Check out repository code"
uses: "actions/checkout@v4"
with:
submodules: true
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: "Setup Python environment"
run: |
pipx install poetry==${{ needs.prepare-environment.outputs.POETRY_VERSION }}
poetry config virtualenvs.create true --local
poetry env use 3.12
- name: "Install dependencies"
run: "poetry install --no-interaction --no-ansi --extras ctl"
- name: "Setup environment"
run: "pip install invoke toml"
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: "npm"
cache-dependency-path: "**/package-lock.json"
- name: Install markdown linter
run: npm install -g markdownlint-cli2
- name: "Generate SDK API documentation"
run: ${{ env.DOCS_COMMAND }}
- name: "Check if SDK API documentation needs to be refreshed"
run: |
git diff --quiet ${SDK_API_DOCS_DIR}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add this as an invoke command too so that it's consistent with how we do it in other repos.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate a bit about which part here you'd add as an invoke command ?
For now, I added an invoke command to generate the API docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean that we can run poetry run invoke generate-sdk-api-docs to generate the docs but then have another command to validate. It could simplify to also have a shortcut for the git diff like we have here:
https://github.com/opsmill/infrahub/blob/infrahub-v1.3.4/tasks/backend.py#L176

This part isn't critical though just a suggestion.


validate-documentation-style:
if: |
always() && !cancelled() &&
Expand All @@ -203,7 +245,7 @@ jobs:
env:
VALE_VERSION: ${{ env.VALE_VERSION }}
- name: "Validate documentation style"
run: ./vale $(find ./docs -type f \( -name "*.mdx" -o -name "*.md" \) )
run: ./vale $(find ./docs -type d -name sdk_ref -prune -o -type f \( -name "*.mdx" -o -name "*.md" \) )

unit-tests:
env:
Expand Down
68 changes: 68 additions & 0 deletions docs/docs/python-sdk/sdk_ref/infrahub_sdk/analyzer.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: analyzer
sidebarTitle: analyzer
---

# `infrahub_sdk.analyzer`

## Classes

### `GraphQLQueryVariable` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L20" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of these links to the main branch along with a line number. Looking at the published docs https://5390d612.infrahub-sdk-python.pages.dev/python-sdk/sdk_ref/infrahub_sdk/analyzer#graphqlqueryvariable- I can't actually see the these links on the page. But they are available in the source. I think we'd need to get rid of them to avoid conflicts as well as having a link that points to an incorrect branch.

If possible it could make sense to have this point to the current version at build time so that we have a link to a tagged release instead of a branch where the line numbers as well as overall structure of the file might change.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it is about docusaurus configuration because the FastMCP doc partake of those links while they don't appear in our documentation.

Mdxify lib provides us with a CLI arg to properly point at the desired git branch (might also work with any git ref but not sure about this) in generated code links. Do you want to leverage this ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say there are two issues the first is that the links are currently broken due to the main branch reference. The second is that as I can't even see the links on the website I don't think they fill a need. It might be because the icon for Github is missing so there's nothing to click on?


### `GraphQLOperation` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L27" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

### `GraphQLQueryAnalyzer` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L32" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

**Methods:**

#### `is_valid` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L40" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
is_valid(self) -> tuple[bool, list[GraphQLError] | None]
```

#### `nbr_queries` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L51" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
nbr_queries(self) -> int
```

#### `operations` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L55" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
operations(self) -> list[GraphQLOperation]
```

#### `contains_mutation` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L68" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
contains_mutation(self) -> bool
```

#### `variables` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L72" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
variables(self) -> list[GraphQLQueryVariable]
```

#### `calculate_depth` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L101" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
calculate_depth(self) -> int
```

Number of nested levels in the query

#### `calculate_height` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L106" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
calculate_height(self) -> int
```

Total number of fields requested in the query

#### `get_fields` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/analyzer.py#L111" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
get_fields(self) -> dict[str, Any]
```
30 changes: 30 additions & 0 deletions docs/docs/python-sdk/sdk_ref/infrahub_sdk/async_typer.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
title: async_typer
sidebarTitle: async_typer
---

# `infrahub_sdk.async_typer`

## Classes

### `AsyncTyper` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/async_typer.py#L11" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

**Methods:**

#### `maybe_run_async` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/async_typer.py#L13" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
maybe_run_async(decorator: Callable, func: Callable) -> Any
```

#### `callback` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/async_typer.py#L25" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
callback(self, *args: Any, **kwargs: Any) -> Any
```

#### `command` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/async_typer.py#L29" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
command(self, *args: Any, **kwargs: Any) -> Any
```
74 changes: 74 additions & 0 deletions docs/docs/python-sdk/sdk_ref/infrahub_sdk/batch.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: batch
sidebarTitle: batch
---

# `infrahub_sdk.batch`

## Functions

### `execute_batch_task_in_pool` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L41" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
execute_batch_task_in_pool(task: BatchTask, semaphore: asyncio.Semaphore, return_exceptions: bool = False) -> tuple[InfrahubNode | None, Any]
```

## Classes

### `BatchTask` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L14" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

### `BatchTaskSync` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L22" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

**Methods:**

#### `execute` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L28" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
execute(self, return_exceptions: bool = False) -> tuple[InfrahubNodeSync | None, Any]
```

Executes the stored task.

### `InfrahubBatch` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L55" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

**Methods:**

#### `num_tasks` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L67" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
num_tasks(self) -> int
```

#### `add` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L70" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
add(self, *args: Any, **kwargs: Any) -> None
```

#### `execute` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L73" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
execute(self) -> AsyncGenerator
```

### `InfrahubBatchSync` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L92" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

**Methods:**

#### `num_tasks` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L99" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
num_tasks(self) -> int
```

#### `add` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L102" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
add(self, *args: Any, **kwargs: Any) -> None
```

#### `execute` <sup><a href="https://github.com/opsmill/infrahub-sdk-python/blob/main/infrahub_sdk/batch.py#L105" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

```python
execute(self) -> Generator[tuple[InfrahubNodeSync | None, Any], None, None]
```
Loading
Loading