Improve IDE/Editor Support for External Accessors

[FBumann]

Is your feature request related to a problem?

External accessor packages have no IDE support:

import xarray as xr
import hvplot.xarray  # required just to register accessor

ds = xr.open_dataset("data.nc")

ds.plot.scatter()    # Built-in: full IDE completion ✓
ds.hvplot.scatter()  # External: nothing. no hints, no docs ✗

Feature	Built-in .plot	External Accessor
Method completion	✓	✗
Parameter hints	✓	✗
Docstrings	✓	✗
Go-to-definition	✓	✗

This affects every accessor package: hvplot, cf-xarray, pint-xarray, rioxarray, xarray-plotly, and the entire xarray-contrib ecosystem.

The current message to package authors is "use the accessor system" - but that system provides a degraded experience compared to built-in features.

Describe the solution you'd like

Typed properties with lazy imports. Same pattern as dask.distributed.

# xarray/core/dataarray.py

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from hvplot.xarray import hvPlotAccessor
    from cf_xarray import CFAccessor

class DataArray:

    @property
    def hvplot(self) -> hvPlotAccessor:
        """hvPlot accessor. Requires: pip install hvplot"""
        from xarray.accessors import load_accessor
        return load_accessor("hvplot", self)

    @property
    def cf(self) -> CFAccessor:
        """CF conventions accessor. Requires: pip install cf-xarray"""
        from xarray.accessors import load_accessor
        return load_accessor("cf", self)

# xarray/accessors.py

ACCESSORS = {
    "hvplot": ("hvplot.xarray", "hvPlotAccessor", "hvplot"),
    "cf": ("cf_xarray", "CFAccessor", "cf-xarray"),
    "pint": ("pint_xarray", "PintDataArrayAccessor", "pint-xarray"),
    "rio": ("rioxarray", "RasterArray", "rioxarray"),
    "plotly": ("xarray_plotly", "DataArrayPlotlyAccessor", "xarray-plotly"),
}

def load_accessor(name: str, obj):
    package, cls_name, install_name = ACCESSORS[name]
    try:
        module = importlib.import_module(package)
        return getattr(module, cls_name)(obj)
    except ImportError:
        raise ImportError(f"{install_name} is not installed.\n\nInstall with: pip install {install_name}")

User experience after:

import xarray as xr

ds.hvplot.scatter()  # just works, full IDE support, no registration import!

Benefits:

• Users: No registration imports, full IDE completion, helpful errors if not installed
• Accessor maintainers: File PR adding ~5 lines, remove register_*_accessor boilerplate
• xarray: Stays lean, implementation remains external, fully backwards compatible

Describe alternatives you've considered

Status quo: Keep register_*_accessor - but IDE support is impossible since types are unknown at static analysis time.

Stub files (.pyi): External packages could ship stubs, but this doesn't solve the "xarray doesn't know about the accessor" problem - the property still needs to exist on DataArray/Dataset.

Additional context

Backwards compatible: Old packages using register_*_accessor continue to work (setattr overwrites property). New packages remove registration and users get IDE completion.

Precedent: This is exactly how dask handles dask.distributed - typed entry point in dask, implementation in separate package.

Origin: PR #11075 - accessor was rejected (correctly) in favor of external package, but this revealed the IDE support gap.

cc: @xarray-contrib @holoviz @corteva

[welcome]

Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the Contributing Guide for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

[jsignell]

Thanks for opening this @FBumann I was doing some experiments and I don't think you need to set the return type explicitly on the property to get the functionality you want. This seems sufficient:

    @property
    def plotly(self):
        if TYPE_CHECKING:
            import xarray_plotly
        else:
            xarray_plotly = attempt_import("xarray_plotly")

        return xarray_plotly.DataArrayPlotlyAccessor

[keewis]

There's just one problem with this proposal: xarray (and I'm guessing pandas has the exact same problem, since that's where the idea of accessors comes from) would need to know about any and all accessors, which are really numerous, which would make the definition of the Dataset, DataArray, and DataTree classes incredibly long. The relationship of dask and distributed is special in that they're always released together, and are maintained by the same team.

With this proposal we're essentially replacing the dynamic nature of accessors (which is good for development and creating new packages, but bad for static typing) with a static relationship (which is good for static analyzers, but bad for development).

In general, I believe that unless we redesign accessors we'd need something like https://mail.python.org/archives/list/typing-sig@python.org/thread/UOIVXOJA2PIEYF3XB37HBI2MAJ4XYNUI/, which proposed allowing to extend the typing using partial stub files. To be clear, this does not appear to exist nor does anyone appear to be working on it, at the moment.

---

If we don't insist on autocompletion using static typing, however, it is possible to use a runtime type checker. beartype, for example (which claims to integrate with pylance... I don't use VSCode myself so I can't verify that), correctly infers the accessor type:

from beartype.bite import infer_hint
import xarray as xr
import cf_xarray

ds = xr.tutorial.open_dataset("air_temperature")
infer_hint(ds.cf)
# result: typing.Annotated[collections.abc.Container, beartype.vale.IsInstance[cf_xarray.accessor.CFDatasetAccessor]]

[shoyer]

If we did this, we could consider requiring that the accessors be maintained in xarray-contrib.

[jsignell]

Yeah I don't think getting rid of register_*_accessor should but the goal, but rather special-casing certain accessors that have a certain level maturity/popularity/maintenance.

[FBumann]

Yeah I don't think getting rid of register_*_accessor should but the goal, but rather special-casing certain accessors that have a certain level maturity/popularity/maintenance.

I would definitely keep the register_*_accessor around!

[FBumann]

Thanks for opening this @FBumann I was doing some experiments and I don't think you need to set the return type explicitly on the property to get the functionality you want. This seems sufficient:

@property
def plotly(self):
    if TYPE_CHECKING:
        import xarray_plotly
    else:
        xarray_plotly = attempt_import("xarray_plotly")

    return xarray_plotly.DataArrayPlotlyAccessor

That would simplify things i guess.

[FBumann]

@keewis
Regarding stub files, i tried to get this working, but couldnt figure it out. But that might just be me, i wasn't even aware of stub files before.
Regarding the class definition, we could move those to another module i guess (see my PR, latest commit)

[FBumann]

Any new on this? @keewis, for me at least code completion is quite important when accessors are used frequently, as it drastically simplifies their usage.

[shoyer]

We discussed adding first-class support for external accessors in the Xarray developer meeting today. This is something that would offers significant accessibility and discoverability benefits for added external packages. It was mentioned that its particularly beneficial in the era of AI coding.

Concerns we discussed:

• How to pick packages. We would need a documented process for how to pick external accessors to bless with first-class support in Xarray:
  • This might be a super-set of the requirements for joining Xarray-Contrib.
  • I would not strictly require that packages be maintained in Xarray-Contrib, because more popular projects tend to want their own GitHub orgs.
  • Probably we want to reserve the right for Xarray maintainers to make an ad-hoc call, possibly requiring some additional evidence of broader adoption, perhaps similar to that recently anounced by JOSS.
• Accessor names pollute a shared namespace. The Dataset namespace is already shared between Xarray's built-in method and user variables (for attribute style look-ups), so some care needs to be taken to minimize the impact. We should consider adding some guidances for good accessor names, e.g.,
  • ✅ .hvplot: this distinctive package name is unlikely to be confused for anything else
  • ✅ .cf: CF is a metadata convention, so this name is not ideal, but is OK for a widely adopted community package like cf-xarray
  • ❌ .crs: Conflicts with the widely used crs variable for coordinate reference systems.
  • ❌ .interpolate: Similar name to built-in Xarray functionality. Prefer clearly distinct names like .myinterp.
• Not a full solution. Accessors developed in private won't be able to make use of this feature, unless they patch Xarray. This is OK, but a limitation worth acknowledging.

From my perspective, despite the costs the benefit would be quite compelling is we talking about adding ~10 external accessors into Xarray. If we're adding ~100 external accessors, then that would add a lot of clutter into Xarray's namespace.

I'd love to hear from @pydata/xarray maintainers (and others) what you think about this proposal, which has long-term maintainability consequences for Xarray.

[keewis]

in particular it might be good to exchange with the pandas devs as we took the concept of accessors from pandas, and thus I would expect pandas to have the same issue.

cc @TomAugspurger, @jorisvandenbossche

[jsignell]

I was thinking more about whether there could be an approach where xarray just provides stubs so that the new functionality is purely for typing. I came across this package: https://pypi.org/project/accessor-stubs/0.0.1/ which seems like a similar idea but it dynamically generates stubs.

I looked around to see if there is an existing xarray-stubs package and it seems like there is not. From glancing at pandas-stubs it seems like a bit of an undertaking... So I was then thinking what if xarray just defined the types of the accessors.

EDIT: I created #11087 to show how this could work