Skip to content

A user tool to debug visibility of objects #522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

A user tool to debug visibility of objects #522

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
46a4ee9
Add initial try at user level debugging for object visibility
robtaylor May 6, 2025
3ac44d8
Add an info log for which modules will be skipped
robtaylor May 6, 2025
86382d9
Add and use HideReason enum
robtaylor Jun 3, 2025
db28216
Use backports.strenum for python <3.11
robtaylor Jun 3, 2025
9c8d0a2
Add tox to dev-dependencies
robtaylor Jun 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 31 additions & 12 deletions autoapi/_mapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
PythonAttribute,
PythonData,
PythonException,
_trace_visibility,
)
from .settings import OWN_PAGE_LEVELS, TEMPLATE_DIR

Expand All @@ -43,6 +44,15 @@ def in_stdlib(module_name: str) -> bool:
LOGGER = sphinx.util.logging.getLogger(__name__)


def _color_info(msg: str) -> None:
LOGGER.info(
colorize("bold", "[AutoAPI] ")
+ colorize(
"darkgreen", msg
)
)


def _expand_wildcard_placeholder(original_module, originals_map, placeholder):
"""Expand a wildcard placeholder to a sequence of named placeholders.

Expand Down Expand Up @@ -329,12 +339,7 @@ def find_files(patterns, dirs, ignore):
for sub_dir in subdirectories.copy():
# iterate copy as we adapt subdirectories during loop
if _path_matches_patterns(os.path.join(root, sub_dir), ignore):
LOGGER.info(
colorize("bold", "[AutoAPI] ")
+ colorize(
"darkgreen", f"Ignoring directory: {root}/{sub_dir}/"
)
)
_color_info(f"Ignoring directory: {root}/{sub_dir}/")
# adapt original subdirectories inplace
subdirectories.remove(sub_dir)
# recurse into remaining directories
Expand All @@ -348,12 +353,7 @@ def find_files(patterns, dirs, ignore):

# Skip ignored files
if _path_matches_patterns(os.path.join(root, filename), ignore):
LOGGER.info(
colorize("bold", "[AutoAPI] ")
+ colorize(
"darkgreen", f"Ignoring file: {root}/{filename}"
)
)
_color_info(f"Ignoring file: {root}/{filename}")
continue

# Make sure the path is full
Expand Down Expand Up @@ -388,6 +388,14 @@ def output_rst(self, source_suffix):
def _output_top_rst(self):
# Render Top Index
top_level_index = os.path.join(self.dir_root, "index.rst")

modules = [obj for obj in self.all_objects.values()
if obj.type == "module" and obj.docstring == ""]
if modules and "undoc-members" not in self.app.config.autoapi_options:
_color_info("The following modules have no top-level documentation, and so were skipped as undocumented:")
for m in modules:
_color_info(f" {m.id}")

pages = [obj for obj in self.objects_to_render.values() if obj.display]
if not pages:
msg = (
Expand Down Expand Up @@ -499,6 +507,7 @@ def _skip_if_stdlib(self):
and not obj["inherited_from"]["is_abstract"]
and module not in documented_modules
):
_trace_visibility(self.app, f"Hiding {obj['qual_name']} as determined to be Python standard Library (found as {obj['full_name']})", verbose=2)
obj["hide"] = True

def _resolve_placeholders(self):
Expand All @@ -518,12 +527,18 @@ def _hide_yo_kids(self):
for module in self.paths.values():
if module["all"] is not None:
all_names = set(module["all"])
_trace_visibility(self.app, f"{module['full_name']}: Found __all__ =")
for n in all_names:
_trace_visibility(self.app, f" {n}")
for child in module["children"]:
if child["qual_name"] not in all_names:
_trace_visibility(self.app, f"Hiding {child['full_name']}, as {child['qual_name']} not in __all__")
child["hide"] = True
elif module["type"] == "module":
_trace_visibility(self.app, f"Testing if any children of {module['full_name']} have already been documented")
for child in module["children"]:
if "original_path" in child:
_trace_visibility(self.app, f"Hiding {child['full_name']} as documented at {child['original_path']}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These items are hidden not because they're documented elsewhere (in fact they may not be documented anywhere) because we don't consider objects imported from the local package into a module to be part of the public API. Usually those imports are used in the implementation of a class or function in that module instead.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, makes sense. I think this caught me out because i use import to populate init, and i wasn't getting documentation because the module had no top level documentation - i.e. the fix was to add a docstring at the top of init. Is this something that should be added to the docs?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does https://sphinx-autoapi.readthedocs.io/en/latest/how_to.html#how-to-customise-what-gets-documented
cover what you're thinking about?

child["hide"] = True

def map(self, options=None):
Expand Down Expand Up @@ -567,13 +582,17 @@ def _render_selection(self):
assert obj.type in self.own_page_types
self.objects_to_render[obj.id] = obj
else:
if obj.subpackages or obj.submodules:
_trace_visibility(self.app, f"Not rendering the following as {obj.id} set to not display:", verbose=2)
for module in itertools.chain(obj.subpackages, obj.submodules):
_trace_visibility(self.app, f" {module.obj['full_name']}", verbose=2)
module.obj["hide"] = True

def _inner(parent):
for child in parent.children:
self.all_objects[child.id] = child
if not parent.display:
_trace_visibility(self.app, f"Hiding {child.id} as parent {parent.id} will not be displayed", verbose=2)
child.obj["hide"] = True

if child.display and child.type in self.own_page_types:
Expand Down
42 changes: 39 additions & 3 deletions autoapi/_objects.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,16 @@
import sphinx.util
import sphinx.util.logging

from sphinx.util.console import colorize

from .settings import OWN_PAGE_LEVELS

LOGGER = sphinx.util.logging.getLogger(__name__)

def _trace_visibility(app, msg: str, verbose=1) -> None:
if app.config.autoapi_verbose_visibility >= verbose:
LOGGER.info(colorize("bold", f"[AutoAPI] [Visibility] {msg}"))


def _format_args(args_info, include_annotations=True, ignore_self=None):
result = []
Expand Down Expand Up @@ -79,6 +85,7 @@ def __init__(
# For later
self._class_content = class_content
self._display_cache: bool | None = None
self._skip_reason = None

def __getstate__(self):
"""Obtains serialisable data for pickling."""
Expand Down Expand Up @@ -199,8 +206,13 @@ def display(self) -> bool:
This attribute depends on the configuration options given in
:confval:`autoapi_options` and the result of :event:`autoapi-skip-member`.
"""
skip = self._should_skip()
if self._display_cache is None:
self._display_cache = not self._ask_ignore(self._should_skip())
self._display_cache = not self._ask_ignore(skip)
if self._display_cache is False:
_trace_visibility(self.app, self._skip_reason)
else:
_trace_visibility(self.app, f"Skipping {self.id} due to cache", verbose=2)

return self._display_cache

Expand Down Expand Up @@ -231,6 +243,22 @@ def _should_skip(self) -> bool:
self.inherited and "inherited-members" not in self.options
)

reason = ""
if self.obj.get("hide", False):
reason = "marked hidden by mapper"
elif skip_undoc_member:
reason = "is undocumented"
elif skip_private_member:
reason = "is a private member"
elif skip_special_member:
reason = "is a special member"
elif skip_imported_member:
reason = "is an imported member"
elif skip_inherited_member:
reason = "is an inherited member"

self._skip_reason = f"Skipping {self.id} as {reason}"

return (
self.obj.get("hide", False)
or skip_undoc_member
Expand All @@ -241,10 +269,16 @@ def _should_skip(self) -> bool:
)

def _ask_ignore(self, skip: bool) -> bool:

ask_result = self.app.emit_firstresult(
"autoapi-skip-member", self.type, self.id, self, skip, self.options
)

if ask_result is not None:
reason = f"Skipping as 'autoapi-skip-member' returned {ask_result}"
if self._skip_reason:
reason += f"Passed skip={skip} to 'autoapi-skip-member' as {self._skip_reason}"
self._skip_reason = reason
return ask_result if ask_result is not None else skip

def _children_of_type(self, type_: str) -> list[PythonObject]:
Expand Down Expand Up @@ -306,11 +340,13 @@ def __init__(self, *args, **kwargs):
"""

def _should_skip(self) -> bool:
return super()._should_skip() or self.name in (
is_new_or_init = self.name in (
"__new__",
"__init__",
)

if not super()._should_skip and is_new_or_init:
self._skip_reason = f"Skipping method {self.id} as is __new__ or __init__"
return super()._should_skip() or is_new_or_init

class PythonProperty(PythonObject):
"""The representation of a property on a class."""
Expand Down
1 change: 1 addition & 0 deletions autoapi/extension.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,7 @@ def setup(app):
app.add_config_value("autoapi_generate_api_docs", True, "html")
app.add_config_value("autoapi_prepare_jinja_env", None, "html")
app.add_config_value("autoapi_own_page_level", "module", "html")
app.add_config_value("autoapi_verbose_visibility", 0, "html")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we utilise logging levels instead of needing to invent our own way of configuring verbosity?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. I was just worrying about not changing the current logging behaviour too much!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, looking at this again, it adds way too much output to debug mode. I think it makes sense to have this separate for debugging visibility. Also I split the visibility debugging into two verbosity levels as usually level 1 will suffice, but allowing for deeper debugging,.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which case please can you add some documentation for this new config option to docs/reference/config.rst. You could also add a small section about debugging to docs/how_to.rst, possibly as a subsection to "How to Customise What Gets Documented", or linking to your new section from there.

app.add_autodocumenter(documenters.AutoapiFunctionDocumenter)
app.add_autodocumenter(documenters.AutoapiPropertyDocumenter)
app.add_autodocumenter(documenters.AutoapiDecoratorDocumenter)
Expand Down
Loading