Merge branch 'master' into compare_tweak

Author: fonnesbeck

.gitignore

@@ -29,3 +29,4 @@ mcmc.sqlite
 benchmarks/env/
 benchmarks/html/
 benchmarks/results/
+.pytest_cache/

.travis.yml

@@ -1,11 +1,11 @@
 language: python
-sudo: false
+sudo: required
 
 cache:
   directories:
     - $HOME/.theano
-    - $HOME/minicoda2
-    - $HOME/minicoda3
+    - $HOME/miniconda2
+    - $HOME/miniconda3
 
 addons:
   apt_packages:
@@ -15,12 +15,11 @@ before_install:
   - . ./scripts/install_miniconda.sh
   - sh -e /etc/init.d/xvfb start
   - export DISPLAY=":99.0"
+  - export MKL_THREADING_LAYER=GNU
 
 install:
   - . ./scripts/create_testenv.sh
-  - pip install coveralls pylint 
-  - pip install travis-sphinx==2.0.0
-  - pip install -r requirements-dev.txt
+  - pip install coveralls travis-sphinx==2.0.0
 
 env:
   - PYTHON_VERSION=2.7 FLOATX='float32' TESTCMD="--durations=10 --ignore=pymc3/tests/test_examples.py --cov-append --ignore=pymc3/tests/test_distributions_random.py --ignore=pymc3/tests/test_variational_inference.py --ignore=pymc3/tests/test_shared.py --ignore=pymc3/tests/test_smc.py --ignore=pymc3/tests/test_updates.py --ignore=pymc3/tests/test_posteriors.py --ignore=pymc3/tests/test_sampling.py"

README.rst

@@ -10,7 +10,7 @@ which focuses on advanced Markov chain Monte Carlo and variational fitting
 algorithms. Its flexibility and extensibility make it applicable to a
 large suite of problems.
 
-Check out the `getting started guide <http://docs.pymc.io/notebooks/getting_started>`__,  or 
+Check out the `getting started guide <http://docs.pymc.io/notebooks/getting_started>`__,  or
 `interact with live examples <https://mybinder.org/v2/gh/pymc-devs/pymc3/master?filepath=%2Fdocs%2Fsource%2Fnotebooks>`__
 using Binder!
 
@@ -114,8 +114,6 @@ Optional
 In addtion to the above dependencies, the GLM submodule relies on
 `Patsy <http://patsy.readthedocs.io/en/latest/>`__.
 
-`scikits.sparse <https://github.com/njsmith/scikits-sparse>`__
-enables sparse scaling matrices which are useful for large problems.
 
 Citing PyMC3
 ============
@@ -127,7 +125,7 @@ in Python using PyMC3. PeerJ Computer Science 2:e55
 Contact
 =======
 
-We are using `discourse.pymc.io <https://discourse.pymc.io/>`__ as our main communication channel. You can also follow us on `Twitter @pymc_devs <https://twitter.com/pymc_devs>`__ for updates and other announcements. 
+We are using `discourse.pymc.io <https://discourse.pymc.io/>`__ as our main communication channel. You can also follow us on `Twitter @pymc_devs <https://twitter.com/pymc_devs>`__ for updates and other announcements.
 
 To ask a question regarding modeling or usage of PyMC3 we encourage posting to our Discourse forum under the `“Questions” Category <https://discourse.pymc.io/c/questions>`__. You can also suggest feature in the `“Development” Category <https://discourse.pymc.io/c/development>`__.
 
@@ -145,11 +143,12 @@ License
 Software using PyMC3
 ====================
 
- - `sampled <https://github.com/ColCarroll/sampled>`__: Decorator for PyMC3.
- - `Bambi <https://github.com/bambinos/bambi>`__: BAyesian Model-Building Interface (BAMBI) in Python.
- - `gelato <https://github.com/ferrine/gelato>`__: Bayesian Neural Networks with PyMC3 and Lasagne.
- - `NiPyMC <https://github.com/PsychoinformaticsLab/nipymc>`__: Bayesian mixed-effects modeling of fMRI data in Python.
- - `beat <https://github.com/hvasbath/beat>`__: Bayesian Earthquake Analysis Tool.
+- `Bambi <https://github.com/bambinos/bambi>`__: BAyesian Model-Building Interface (BAMBI) in Python.
+- `pymc3_models <https://github.com/parsing-science/pymc3_models>`__: Custom PyMC3 models built on top of the scikit-learn API.
+- `webmc3 <https://github.com/AustinRochford/webmc3>`__: A web interface for exploring PyMC3 traces
+- `sampled <https://github.com/ColCarroll/sampled>`__: Decorator for PyMC3 models.
+- `NiPyMC <https://github.com/PsychoinformaticsLab/nipymc>`__: Bayesian mixed-effects modeling of fMRI data in Python.
+- `beat <https://github.com/hvasbath/beat>`__: Bayesian Earthquake Analysis Tool.
 
 
 Please contact us if your software is not listed here.
@@ -177,7 +176,7 @@ Sponsors
 
 |Quantopian|
 
-.. |Binder| image:: https://mybinder.org/badge.svg 
+.. |Binder| image:: https://mybinder.org/badge.svg
    :target: https://mybinder.org/v2/gh/pymc-devs/pymc3/master?filepath=%2Fdocs%2Fsource%2Fnotebooks
 .. |Build Status| image:: https://travis-ci.org/pymc-devs/pymc3.png?branch=master
    :target: https://travis-ci.org/pymc-devs/pymc3

RELEASE-NOTES.md

@@ -1,6 +1,39 @@
 # Release Notes
 
-## PyMC 3.3. (Unreleased) 
+## PyMC 3.4 (unreleased)
+
+### New features
+
+- Add `logit_p` keyword to `pm.Bernoulli`, so that users can specify the logit of the success probability. This is faster and more stable than using `p=tt.nnet.sigmoid(logit_p)`.
+- Add `random` keyword to `pm.DensityDist` thus enabling users to pass custom random method which in turn makes sampling from a `DensityDist` possible.
+- Effective sample size computation is updated. The estimation uses Geyer's initial positive sequence, which no longer truncates the autocorrelation series inaccurately. `pm.diagnostics.effective_n` now can reports N_eff>N.
+- Added `KroneckerNormal` distribution and a corresponding `MarginalKron`
+  Gaussian Process implementation for efficient inference, along with
+  lower-level functions such as `cartesian` and `kronecker` products.
+- Added `Coregion` covariance function.
+- Add new 'pairplot' function, for plotting scatter or hexbin matrices of sampled parameters.
+  Optionally it can plot divergences.
+- Plots of discrete distributions in the docstrings
+- Add logitnormal distribution
+- Densityplot: add support for discrete variables
+- Fix the Binomial likelihood in `.glm.families.Binomial`, with the flexibility of specifying the `n`. 
+- Add `offset` kwarg to `.glm`.
+
+### Fixes
+
+- `VonMises` does not overflow for large values of kappa. i0 and i1 have been removed and we now use log_i0 to compute the logp.
+- The bandwidth for KDE plots is computed using a modified version of Scott's rule. The new version uses entropy instead of standard deviation. This works better for multimodal distributions. Functions using KDE plots has a new argument `bw` controlling the bandwidth.
+- fix PyMC3 variable is not replaced if provided in more_replacements (#2890)
+
+### Deprecations
+
+- DIC and BPIC calculations have been removed
+- df_summary have been removed, use summary instead
+- `njobs` and `nchains` kwarg are deprecated in favor of `cores` and `chains` for `sample`
+- `lag` kwarg in `pm.stats.autocorr` and `pm.stats.autocov` is deprecated.
+
+
+## PyMC 3.3 (January 9, 2018)
 
 ### New features
 
@@ -10,13 +43,12 @@
 - New benchmark suite added (see http://pandas.pydata.org/speed/pymc3/)
 - Generalized random seed types
 - Update loo, new improved algorithm (#2730)
-- New CSG (Constant Stochastic Gradient) approximate posterior sampling
-  algorithm (#2544)
+- New CSG (Constant Stochastic Gradient) approximate posterior sampling algorithm (#2544)
 - Michael Osthege added support for population-samplers and implemented differential evolution metropolis (`DEMetropolis`).  For models with correlated dimensions that can not use gradient-based samplers, the `DEMetropolis` sampler can give higher effective sampling rates. (also see [PR#2735](https://github.com/pymc-devs/pymc3/pull/2735))
 - Forestplot supports multiple traces (#2736)
 - Add new plot, densityplot (#2741)
 - DIC and BPIC calculations have been deprecated
-- `compare` now accepts dictionary of model:trace pairs instead of lists
+- Refactor HMC and implemented new warning system (#2677, #2808) 
 
 ### Fixes
 
@@ -28,6 +60,12 @@
 - Fixed `sample_ppc` and `sample_ppc_w` to iterate all chains(#2633, #2748)
 - Add Bayesian R2 score (for GLMs) `stats.r2_score` (#2696) and test (#2729).
 - SMC works with transformed variables (#2755)
+- Speedup OPVI (#2759)
+- Multiple minor fixes and improvements in the docs (#2775, #2786, #2787, #2789, #2790, #2794, #2799, #2809)
+
+### Deprecations
+
+- Old (`minibatch-`)`advi` is removed (#2781)
 
 
 ## PyMC3 3.2 (October 10, 2017)

benchmarks/benchmarks/benchmarks.py

@@ -71,7 +71,8 @@ def setup(self, step):
 
     def time_overhead_sample(self, step):
         with self.model:
-            pm.sample(self.n_steps, step=step(), random_seed=1)
+            pm.sample(self.n_steps, step=step(), random_seed=1,
+                      progressbar=False, compute_convergence_checks=False)
 
 
 class ExampleSuite(object):
@@ -115,11 +116,13 @@ def time_drug_evaluation(self):
             pm.Deterministic('difference of stds', group1_std - group2_std)
             pm.Deterministic(
                 'effect size', diff_of_means / np.sqrt((group1_std**2 + group2_std**2) / 2))
-            pm.sample(20000, njobs=4, chains=4)
+            pm.sample(draws=20000, cores=4, chains=4,
+                      progressbar=False, compute_convergence_checks=False)
 
     def time_glm_hierarchical(self):
         with glm_hierarchical_model():
-            pm.sample(draws=20000, njobs=4, chains=4)
+            pm.sample(draws=20000, cores=4, chains=4,
+                      progressbar=False, compute_convergence_checks=False)
 
 
 class NUTSInitSuite(object):
@@ -141,8 +144,9 @@ def track_glm_hierarchical_ess(self, init):
         with glm_hierarchical_model():
             start, step = pm.init_nuts(init=init, chains=self.chains, progressbar=False, random_seed=123)
             t0 = time.time()
-            trace = pm.sample(draws=self.draws, step=step, njobs=4, chains=self.chains,
-                              start=start, random_seed=100)
+            trace = pm.sample(draws=self.draws, step=step, cores=4, chains=self.chains,
+                              start=start, random_seed=100, progressbar=False,
+                              compute_convergence_checks=False)
             tot = time.time() - t0
         ess = pm.effective_n(trace, ('mu_a',))['mu_a']
         return ess / tot
@@ -154,8 +158,9 @@ def track_marginal_mixture_model_ess(self, init):
                                    progressbar=False, random_seed=123)
             start = [{k: v for k, v in start.items()} for _ in range(self.chains)]
             t0 = time.time()
-            trace = pm.sample(draws=self.draws, step=step, njobs=4, chains=self.chains,
-                              start=start, random_seed=100)
+            trace = pm.sample(draws=self.draws, step=step, cores=4, chains=self.chains,
+                              start=start, random_seed=100, progressbar=False,
+                              compute_convergence_checks=False)
             tot = time.time() - t0
         ess = pm.effective_n(trace, ('mu',))['mu'].min()  # worst case
         return ess / tot
@@ -178,8 +183,9 @@ def track_glm_hierarchical_ess(self, step):
             if step is not None:
                 step = step()
             t0 = time.time()
-            trace = pm.sample(draws=self.draws, step=step, njobs=4, chains=4,
-                              random_seed=100)
+            trace = pm.sample(draws=self.draws, step=step, cores=4, chains=4,
+                              random_seed=100, progressbar=False,
+                              compute_convergence_checks=False)
             tot = time.time() - t0
         ess = pm.effective_n(trace, ('mu_a',))['mu_a']
         return ess / tot

docs/source/_static/.gitignore

@@ -7,7 +7,7 @@ Using shared variables
 
 Shared variables allow us to use values in theano functions that are
 not considered an input to the function, but can still be changed
-later. They are very similar to global variables in may ways.::
+later. They are very similar to global variables in may ways::
 
     a = tt.scalar('a')
     # Create a new shared variable with initial value of 0.1
@@ -23,7 +23,7 @@ their shape as long as the number of dimensions stays the same.
 
 We can use shared variables in PyMC3 to fit the same model to several
 datasets without the need to recreate the model each time (which can
-be time consuming if the number of datasets is large).::
+be time consuming if the number of datasets is large)::
 
     # We generate 10 datasets
     true_mu = [np.random.randn() for _ in range(10)]

docs/source/advanced_theano.rst

@@ -18,3 +18,4 @@ API Reference
    api/backends
    api/math
    api/data
+   api/model

docs/source/api.rst

@@ -2,17 +2,21 @@
 Bounded Variables
 =================
 
-PyMC3 includes the construct ``Bound`` for placing constraints on existing
-probability distributions.  It modifies a given distribution to take values
-only within a specified interval.  
+PyMC3 includes the construct :class:`~pymc3.distributions.bound.Bound` for
+placing constraints on existing probability distributions.  It modifies a given
+distribution to take values only within a specified interval.
 
 Some types of variables require constraints.  For instance, it doesn't make
 sense for a standard deviation to have a negative value, so something like a
 Normal prior on a parameter that represents a standard deviation would be
 inappropriate.  PyMC3 includes distributions that have positive support, such
-as ``Gamma`` or ``Exponential``.  PyMC3 also includes several bounded
-distributions, such as ``Uniform``, ``HalfNormal``, and ``HalfCauchy``, that
-are restricted to a specific domain.  
+as :class:`~pymc3.distributions.continuous.Gamma` or
+:class:`~pymc3.distributions.continuous.Exponential`.  PyMC3 also includes
+several bounded distributions, such as
+:class:`~pymc3.distributions.continuous.Uniform`,
+:class:`~pymc3.distributions.continuous.HalfNormal`, and
+:class:`~pymc3.distributions.continuous.HalfCauchy`, that are restricted to a
+specific domain.
 
 All univariate distributions in PyMC3 can be given bounds.  The distribution of
 a continuous variable that has been bounded is automatically transformed into
@@ -32,7 +36,7 @@ specification of a bounded distribution should go within the model block::
     with pm.Model() as model:
         BoundedNormal = pm.Bound(pm.Normal, lower=0.0)
         x = BoundedNormal('x', mu=1.0, sd=3.0)
-        
+
 If the bound will be applied to a single variable in the model, it may be
 cleaner notationally to define both the bound and variable together. ::
 
@@ -42,17 +46,24 @@ cleaner notationally to define both the bound and variable together. ::
 Bounds can also be applied to a vector of random variables.  With the same
 ``BoundedNormal`` object we created previously we can write::
 
-    with model: 
+    with model:
         x_vector = BoundedNormal('x_vector', mu=1.0, sd=3.0, shape=3)
 
 Caveats
 #######
 
 * Bounds cannot be given to variables that are ``observed``.  To model
-  truncated data, use a ``Potential`` in combination with a cumulative
+  truncated data, use a :func:`~pymc3.model.Potential` in combination with a cumulative
   probability function.  See `this example <https://github.com/pymc-devs/pymc3/blob/master/pymc3/examples/censored_data.py>`_.
 
 * The automatic transformation applied to continuous distributions results in
   an unnormalized probability distribution.  This doesn't effect inference
   algorithms but may complicate some model comparison procedures.
 
+API
+###
+
+
+.. currentmodule:: pymc3.distributions.bound
+.. automodule:: pymc3.distributions.bound
+   :members:

docs/source/api/bounds.rst

@@ -29,6 +29,8 @@ Continuous
    VonMises
    Triangular
    Gumbel
+   Logistic
+   LogitNormal
    Interpolated
 
 .. automodule:: pymc3.distributions.continuous