feature: include proper labels when inspecting compiled module

[elfkuzco]

Is your feature request related to a problem? Please describe.

Griffe does not show correct labels for @property when forcing inspection as opposed to when using just .pyi files. As a consequence, one has to write the docstrings just like for regular function when a concise line like str: xxxxxxx (google style) would suffice to describe the attribute. Also, this causes poor documentation as the docs is filled with <attribute of 'module.member' objects

Describe the solution you'd like

I would like the ability to correctly infer labels of attributes from compiled modules.

Describe alternatives you've considered

I have not considered any other alternative solution/feature. I am fairly new to building documentation with griffe and while I don't know if leveraging the extension route would help, I feel this would be a great addition to the tool.

Additional context

See comparison of the attributes when inspecting a compiled module vs when using the .pyi files.

[elfkuzco]

I just realized this should have been in the mkdocstrings/python repo instead. Creating it there instead. Sorry.

[pawamoy]

It's actually more relevant to Griffe, lets use this issue 🙂

[pawamoy]

Thanks for the request @elfkuzco 🙂 I think it's a bug rather than a feature request: Griffe is supposed to correctly infer that a property is a property, even when using dynamic analysis.

Could you provide a minimal reproduction example? It's hard to tell why Griffe sees the property as a class instead of a property without the compiled module at hand.

[elfkuzco]

To Reproduce:

git clone git@github.com:elfkuzco/python-libzim.git -b online-api-doc
hatch shell
SIGN_APPLE=0 hatch run build-ext
# generate the graph with forced analysis
griffe dump -x libzim/libzim.so > libzim.forced.json
# generate the graph without forced analaysis
griffe dump libzim > libzim.json

Rather than viewing the files to confirm that the @property is missing:

# Search for propery labels in compiled module
griffe dump -x libzim/libzim.so | grep property
# Search for propery labels without forced analysis
griffe dump  libzim | grep property

I also added the -r flag to be sure but it turns out to be the same. The graph with and without dynamic analysis are different.

[pawamoy]

Cython is doing something I don't understand. The libzim.libzim module (at path libzim/libzim.so) has the module type, but doesn't expose anything. Its repr shows <module 'libzim.libzim' (<class 'libzim.ModuleLoader'>)>. I'm not sure to understand what this libzim.ModuleLoader is and whether it plays a role.

The other related thing that is bugging me is that your libzim top-level package doesn't have an __init__.py file, only an __init__.pyi one, and instead of importing submodules (reader, writer, etc.) from libzim.libzim (the compiled file), it imports from libzim (top-level). That doesn't make sense to me, but again, I'm not sure what Cython is doing under the hood.

IMO __init__.pyi should be renamed __init__.py (otherwise how users are even able to import anything from it?), and it should import from libzim.libzim:

from libzim.libzim import (
    reader,  # noqa: F401 # pyright: ignore[reportUnusedImport]
    search,  # noqa: F401 # pyright: ignore[reportUnusedImport]
    suggestion,  # noqa: F401 # pyright: ignore[reportUnusedImport]
    version,  # noqa: F401 # pyright: ignore[reportUnusedImport]
    writer,  # noqa: F401 # pyright: ignore[reportUnusedImport]
)

Looks like I'm out of my depth here as I don't understand how it even works 🙂 Cython seems to be using some dynamic loading mechanisms that I'm not aware of. If I build the package with hatch build, then extract the wheel, then as I mentioned above, without __init__.py the libzim package is a namespace package and it doesn't expose anything (no submodule, nothing).

I do note that running griffe dump libzim/libzim.so produces results, but maybe only because it imports it (since it's compiled) and the dynamic loading mechanism offered by Cython/setuptools kicks in (because it's installed in editable mode).

To make things more difficult, VSCode's debugger crashes when Griffe tries to import libzim.libzim using importlib.import_module 😅

In the end, I was able to debug this, and the type of Archive.uuid is <class 'getset_descriptor'>. The fix should therefore be to add a condition to check for this type and a method in the inspector to handle it as a property 🙂

[pawamoy]

Yeah OK there's indeed a .pth file and associated module in the venv, which uses a meta path finder. Still, to me it will only work when installed in editable mode, not when distributed 🤔