|
3 | 3 |  |
4 | 4 |  |
5 | 5 | [](https://opensource.org/licenses/Apache-2.0) |
| 6 | +<img src="https://github.com/StatMixedML/LightGBMLSS/actions/workflows/mkdocs.yaml/badge.svg" alt="Documentation status badge"> |
6 | 7 | <img src="https://github.com/StatMixedML/LightGBMLSS/workflows/unit-tests/badge.svg" alt="Unit test status badge"> |
7 | 8 | <img src="https://codecov.io/gh/StatMixedML/LightGBMLSS/branch/master/graph/badge.svg" alt="Code coverage status badge"> |
8 | 9 |
|
9 | 10 | </h4> |
10 | 11 |
|
11 | 12 | # |
12 | | -<img align="right" width="156.5223" height="181.3" src="../master/figures/LightGBMLSS.png"> |
| 13 | +<img align="right" width="156.5223" height="181.3" src="figures/LightGBMLSS.png"> |
13 | 14 |
|
14 | 15 |
|
15 | 16 | # LightGBMLSS - An extension of LightGBM to probabilistic modelling and prediction |
16 | | -We propose a new framework of LightGBM that predicts the entire conditional distribution of a univariate response variable. In particular, **LightGBMLSS** models all moments of a parametric distribution, i.e., mean, location, scale and shape (LSS), instead of the conditional mean only. Choosing from a wide range of continuous, discrete, and mixed discrete-continuous distributions, modelling and predicting the entire conditional distribution greatly enhances the flexibility of LightGBM, as it allows to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived. |
| 17 | +We introduce a comprehensive framework that models and predicts the full conditional distribution of univariate targets as a function of covariate. Choosing from a wide range of continuous, discrete, and mixed discrete-continuous distributions, modelling and predicting the entire conditional distribution greatly enhances the flexibility of LightGBM, as it allows to create probabilistic forecasts from which prediction intervals and quantiles of interest can be derived. |
17 | 18 |
|
18 | 19 | ## Features |
19 | 20 | :white_check_mark: Estimation of all distributional parameters. <br/> |
@@ -46,48 +47,18 @@ Then, to install the shap-dependency, run |
46 | 47 | pip install git+https://github.com/dsgibbons/shap.git |
47 | 48 | ``` |
48 | 49 |
|
49 | | -## Available Distributions |
50 | | -LightGBMLSS currently supports the following distributions. |
| 50 | +## `Available Distributions` |
| 51 | +Our framework is built upon PyTorch and Pyro, enabling users to harness a diverse set of distributional families. LightGBMLSS currently supports the [following distributions](https://statmixedml.github.io/LightGBMLSS/distributions/). |
51 | 52 |
|
52 | | -| Distribution | Usage |Type | Support | Number of Parameters | |
53 | | -| :----------------------------------------------------------------------------------------------------------------------------------: |:------------------------: |:-------------------------------------: | :-----------------------------: | :-----------------------------: | |
54 | | -| [Beta](https://pytorch.org/docs/stable/distributions.html#beta) | `Beta()` | Continuous <br /> (Univariate) | $y \in (0, 1)$ | 2 | |
55 | | -| [Cauchy](https://pytorch.org/docs/stable/distributions.html#cauchy) | `Cauchy()` | Continuous <br /> (Univariate) | $y \in (-\infty,\infty)$ | 2 | |
56 | | -| [Expectile](https://epub.ub.uni-muenchen.de/31542/1/1471082x14561155.pdf) | `Expectile()` | Continuous <br /> (Univariate) | $y \in (-\infty,\infty)$ | Number of expectiles | |
57 | | -| [Gamma](https://pytorch.org/docs/stable/distributions.html#gamma) | `Gamma()` | Continuous <br /> (Univariate) | $y \in (0, \infty)$ | 2 | |
58 | | -| [Gaussian](https://pytorch.org/docs/stable/distributions.html#normal) | `Gaussian()` | Continuous <br /> (Univariate) | $y \in (-\infty,\infty)$ | 2 | |
59 | | -| [Gumbel](https://pytorch.org/docs/stable/distributions.html#gumbel) | `Gumbel()` | Continuous <br /> (Univariate) | $y \in (-\infty,\infty)$ | 2 | |
60 | | -| [Laplace](https://pytorch.org/docs/stable/distributions.html#laplace) | `Laplace()` | Continuous <br /> (Univariate) | $y \in (-\infty,\infty)$ | 2 | |
61 | | -| [LogNormal](https://pytorch.org/docs/stable/distributions.html#lognormal) | `LogNormal()` | Continuous <br /> (Univariate) | $y \in (0,\infty)$ | 2 | |
62 | | -| [Negative Binomial](https://pytorch.org/docs/stable/distributions.html#negativebinomial) | `NegativeBinomial()` | Discrete Count <br /> (Univariate) | $y \in (0, 1, 2, 3, \ldots)$ | 2 | |
63 | | -| [Poisson](https://pytorch.org/docs/stable/distributions.html#poisson) | `Poisson()` | Discrete Count <br /> (Univariate) | $y \in (0, 1, 2, 3, \ldots)$ | 1 | |
64 | | -| [Spline Flow](https://docs.pyro.ai/en/stable/distributions.html#pyro.distributions.transforms.Spline) | `SplineFlow()` | Continuous \& Discrete Count <br /> (Univariate) | $y \in (-\infty,\infty)$ <br /> <br /> $y \in [0, \infty)$ <br /> <br /> $y \in [0, 1]$ <br /> <br /> $y \in (0, 1, 2, 3, \ldots)$ | 2xcount_bins + (count_bins-1) (order=quadratic) <br /> <br /> 3xcount_bins + (count_bins-1) (order=linear) | |
65 | | -| [Student-T](https://pytorch.org/docs/stable/distributions.html#studentt) | `StudentT()` | Continuous <br /> (Univariate) | $y \in (-\infty,\infty)$ | 3 | |
66 | | -| [Weibull](https://pytorch.org/docs/stable/distributions.html#weibull) | `Weibull()` | Continuous <br /> (Univariate) | $y \in [0, \infty)$ | 2 | |
67 | | -| [Zero-Adjusted Beta](https://github.com/pyro-ppl/pyro/blob/dev/pyro/distributions/zero_inflated.py) | `ZABeta()` | Discrete-Continuous <br /> (Univariate) | $y \in [0, 1)$ | 3 | |
68 | | -| [Zero-Adjusted Gamma](https://github.com/pyro-ppl/pyro/blob/dev/pyro/distributions/zero_inflated.py) | `ZAGamma()` | Discrete-Continuous <br /> (Univariate) | $y \in [0, \infty)$ | 3 | |
69 | | -| [Zero-Adjusted LogNormal](https://github.com/pyro-ppl/pyro/blob/dev/pyro/distributions/zero_inflated.py) | `ZALN()` | Discrete-Continuous <br /> (Univariate) | $y \in [0, \infty)$ | 3 | |
70 | | -| [Zero-Inflated Negative Binomial](https://github.com/pyro-ppl/pyro/blob/dev/pyro/distributions/zero_inflated.py#L150) | `ZINB()` | Discrete-Count <br /> (Univariate) | $y \in [0, 1, 2, 3, \ldots)$ | 3 | |
71 | | -| [Zero-Inflated Poisson](https://github.com/pyro-ppl/pyro/blob/dev/pyro/distributions/zero_inflated.py#L121) | `ZIPoisson()` | Discrete-Count <br /> (Univariate) | $y \in [0, 1, 2, 3, \ldots)$ | 2 | |
| 53 | +## `How to use` |
| 54 | +Please visit the [example section](https://statmixedml.github.io/LightGBMLSS/examples/Gaussian_Regression/) for guidance on how to use the framework. |
72 | 55 |
|
73 | | -## Some Notes |
74 | | -> ### Stabilization |
75 | | -Since LightGBMLSS updates the parameter estimates by optimizing Gradients and Hessians, it is important that these are comparable in magnitude for all distributional parameters. Due to variability regarding the ranges, the estimation of Gradients and Hessians might become unstable so that LightGBMLSS might not converge or might converge very slowly. To mitigate these effects, we have implemented a stabilization of Gradients and Hessians. |
| 56 | +## `Documentation` |
| 57 | +For more information and context, please visit the [documentation](https://statmixedml.github.io/LightGBMLSS/). |
76 | 58 |
|
77 | | -For improved convergence, an alternative approach is to standardize the (continuous) response variable, such as dividing it by 100 (e.g., y/100). This approach proves especially valuable when the response range significantly differs from that of Gradients and Hessians. Nevertheless, it is essential to carefully evaluate and apply both the built-in stabilization and response standardization techniques in consideration of the specific dataset at hand. |
| 59 | +## `Feedback` |
| 60 | +We encourage you to provide feedback on how to enhance LightGBMLSS or request the implementation of additional distributions by opening a [new discussion](https://github.com/StatMixedML/LightGBMLSS/discussions). |
78 | 61 |
|
79 | | -> ### Runtime |
80 | | -Since LightGBMLSS is based on a *one vs. all estimation strategy*, where a separate tree is grown for each distributional parameter, it requires training ```[number of iterations] * [number of distributional parameters]``` trees. Hence, the runtime of LightGBMLSS is generally slightly higher for univariate distributions as compared to LightGBM, which requires training ```[number of iterations]``` trees only. |
81 | | - |
82 | | -## Feedback |
83 | | -We encourage you to provide feedback on how to enhance LightGBMLSS or request the implementation of additional distributions by opening a new discussion. |
84 | | - |
85 | | -## Reference Paper |
86 | | -März, A. and Kneib, T.: (2022) [*Distributional Gradient Boosting Machines*](https://arxiv.org/abs/2204.00778). <br/> |
87 | | -März, Alexander (2019): [*XGBoostLSS - An extension of XGBoost to probabilistic forecasting*](https://arxiv.org/abs/1907.03178). |
88 | | - |
89 | | - |
90 | | -<!--- |
| 62 | +## `Reference Paper` |
91 | 63 | [](https://arxiv.org/abs/2204.00778) <br/> |
92 | 64 | [](https://arxiv.org/abs/1907.03178) <br/> |
93 | | ----> |
0 commit comments