SPEI: Passing rv_continuous distributions

[rubhae]

Setup Information

• Xclim version: 0.53.2
• Python version: 3.12.7
• Operating System: Ubuntu 24.04

Description

I tried to calculate SPEI with a GenEV distribution as suggested in Stagge et al. 2015.

The documentation of xclim.indices.standardized_precipitation_evapotranspiration_index claims that rv_continuous can be used for the distribution:

dist ({‘gamma’, ‘fisk’} or rv_continuous function) – Name of the univariate distribution, or a callable rv_continuous (see scipy.stats).

However, this does not work, as only the below shown manually written set of distributions can be used.
In xclim.indices.stats.standardized_index_fit_params, the following check makes it impossible to use any other distribution:

dist_and_methods = {"gamma": ["ML", "APP", "PWM"], "fisk": ["ML", "APP"]}
dist = xclim.indices.stats.get_dist(dist)
if dist.name not in dist_and_methods:
    raise NotImplementedError(f"The distribution `{dist.name}` is not supported.")
if method not in dist_and_methods[dist.name]:
    raise NotImplementedError(
        f"The method `{method}` is not supported for distribution `{dist.name}`."

Steps To Reproduce

If I try to run:

spei_gev = xclim.indices.standardized_precipitation_evapotranspiration_index(
    wb,
    freq="MS",
    window=1,
    dist=scipy.stats.genextreme,
    method="ML",
)

I get the error message:

NotImplementedError: <scipy.stats._continuous_distns.genextreme_gen object at 0x77859811b2f0> distribution is not yet implemented.

Additional context

I was able to use scipy.stats.genextreme as a distribution by bypassing the if statement above by only checking against the dictionary dist_and_methods if "dist" is a string:

dist_and_methods = {"gamma": ["ML", "APP", "PWM"], "fisk": ["ML", "APP"]}
if isinstance(dist, str):
    dist = xclim.indices.stats.get_dist(dist)
    if dist.name not in dist_and_methods:
        raise NotImplementedError(f"The distribution `{dist.name}` is not supported.")
    if method not in dist_and_methods[dist.name]:
        raise NotImplementedError(
            f"The method `{method}` is not supported for distribution `{dist.name}`."

The same method is used in standardized_precipitation_evapotranspiration_index().

Contribution

• I would be willing/able to open a Pull Request to address this bug.

Code of Conduct

• I agree to follow this project's Code of Conduct

[aulemahal]

Hi @rubhae !

Thanks for the issue, indeed, passing a rv_continuous object seems like an "expert" usage and the function shouldn't fail on this simple check.

For you information, more recent versions of xclim have changed this part of the code, and I think your usecase would now work (genextreme and ML). But still, your suggestion looks sensible.

We could relax the test:

xclim/src/xclim/indices/stats.py

Line 859 in 675fb4d

if method != "PWM":

to also allow any distributions if method = "ML" ? Or maybe transform the errors into warnings.

The reason xclim says its "not implemented" is that we have custom algorithms to have a good first guess of the fit and accelerate/improve the "ML" fit. But these algorithms are limited to a handful of named methods, which is why I think we should still emit a warning when an unknown method is passed.

Thoughts of @coxipi ?

[coxipi]

Hi! Yes, the rv_continuous bit was specifically added to allow the use of lmoments3, which cannot be included as-is for licensing issues. The restrictions we had are for distributions are specifically tested and I knew were used in the litterature for SPI/SPEI, so it could make sense to add GEV to SPEI if this is used by Stagge,2015. We already have a _fit_start for this distribution, I don't know how well it works. More generally, I think allowing other rv_continuous (more generally, with any fitting method) as an expert mode makes sense.

[coxipi]

Actually, we had left the possibility of using rv_continuous, so this is just a bug. It's weird, I would think test_str_vs_rv_continuous should fail in because of this