ENH: Add Dockerfile for development inside a container (#1527)

* Add Dockerfile for development inside a container

* Use /Users/austin for portability

* Put Dockerfile with scripts

* Add script for starting development Docker container

* Really add script this time

* Modify options to create_testenv.sh

* Shift while parsing arguments

* Update dockerfile to run test suite

* Add documentation for docker

* Typo

Author: twiecki

.gitignore

@@ -21,3 +21,6 @@ _build
 
 # Test artifacts
 mcmc.sqlite
+
+# Docker development
+notebooks/

CONTRIBUTING.md

@@ -11,17 +11,17 @@ There are 4 main ways of contributing to the PyMC3 project (in descending order
 
 # Opening issues
 
-We appreciate being notified of problems with the existing PyMC code. We prefer that issues be filed the on [Gitub Issue Tracker](https://github.com/pymc-devs/pymc3/issues), rather than on social media or by direct email to the developers. 
+We appreciate being notified of problems with the existing PyMC code. We prefer that issues be filed the on [Gitub Issue Tracker](https://github.com/pymc-devs/pymc3/issues), rather than on social media or by direct email to the developers.
 
 Please verify that your issue is not being currently addressed by other issues or pull requests by using the GitHub search tool to look for key words in the project issue tracker.
 
 # Contributing code via pull requests
 
-While issue reporting is valuable, we strongly encourage users who are inclined to do so to submit patches for new or existing issues via pull requests. This is particularly the case for simple fixes, such as typos or tweaks to documentation, which do not require a heavy investment of time and attention. 
+While issue reporting is valuable, we strongly encourage users who are inclined to do so to submit patches for new or existing issues via pull requests. This is particularly the case for simple fixes, such as typos or tweaks to documentation, which do not require a heavy investment of time and attention.
 
 Contributors are also encouraged to contribute new code to enhance PyMC's functionality, also via pull requests. Please consult the [PyMC3 documentation](https://pymc-devs.github.io/pymc3/) to ensure that any new contribution does not strongly overlap with existing functionality.
 
-The preferred workflow for contributing to PyMC3 is to fork the [GitHUb repository](https://github.com/pymc-devs/pymc3/), clone it to your local machine, and develop on a feature branch. 
+The preferred workflow for contributing to PyMC3 is to fork the [GitHUb repository](https://github.com/pymc-devs/pymc3/), clone it to your local machine, and develop on a feature branch.
 
 ## Steps:
 
@@ -33,13 +33,13 @@ The preferred workflow for contributing to PyMC3 is to fork the [GitHUb reposito
    $ git clone [email protected]:<your GitHub handle>/pymc3.git
    $ cd pymc3-learn
    ```
-   
+
 3. Create a ``feature`` branch to hold your development changes:
 
    ```bash
    $ git checkout -b my-feature
    ```
-   
+
    Always use a ``feature`` branch. It's good practice to never routinely work on the ``master`` branch of any repository.
 
 4. Develop the feature on your feature branch. Add changed files using ``git add`` and then ``git commit`` files:
@@ -63,11 +63,12 @@ We recommended that your contribution complies with the following guidelines bef
 
 *  If your pull request addresses an issue, please use the pull request title to describe the issue and mention the issue number in the pull request description. This will make sure a link back to the original issue is created.
 
-*  All public methods must have informative docstrings with sample usage presented as doctests when appropriate.
+*  All public methods must have informative docstrings with sample usage when appropriate.
 
 *  Please prefix the title of incomplete contributions with `[WIP]` (to indicate a work in progress). WIPs may be useful to (1) indicate you are working on something to avoid duplicated work, (2) request broad review of functionality or API, or (3) seek collaborators.
 
-*  All other tests pass when everything is rebuilt from scratch. 
+*  All other tests pass when everything is rebuilt from scratch.  See
+[Developing in Docker](#Developing-in-Docker) for information on running the test suite locally.
 
 *  When adding additional functionality, provide at least one example script or Jupyter Notebook in the ``pymc3/examples/`` folder. Have a look at other examples for reference. Examples should demonstrate why the new functionality is useful in practice and, if possible, compare it to other methods available in PyMC3.
 
@@ -104,9 +105,27 @@ tools:
   $ autopep8 path/to/pep8.py
   ```
 
+## Developing in Docker
+
+We have provided a Dockerfile which helps for isolating build problems, and local development.
+Install [Docker](https://www.docker.com/) for your operating system, clone this repo, then
+run `./scripts/start_container.sh`. This should start a local docker container called `pymc3`,
+as well as a [`jupyter`](http://jupyter.org/) notebook server running on port 8888.  The repo
+will be running the code from your local copy of `pymc3`, so it is good for development.  You may
+also use it to run the test suite, with
+
+```bash
+$  docker exec -it pymc3  bash # logon to the container
+$  cd ~/pymc3  
+$  . ./scripts/test.sh # takes a while!
+```
+
+This should be quite close to how the tests run on TravisCI.
+
+
 ## Style guide
 
 Follow [TensorFlow's style guide](https://www.tensorflow.org/versions/master/how_tos/style_guide.html) or the [Google style guide](https://google.github.io/styleguide/pyguide.html) for writing code, which more or less follows PEP 8.
 
 
-#### This guide was derived from the [scikit-learn guide to contributing](https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md) 
+#### This guide was derived from the [scikit-learn guide to contributing](https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md)

scripts/Dockerfile

@@ -0,0 +1,16 @@
+FROM jupyter/minimal-notebook
+
+MAINTAINER Austin Rochford <[email protected]>
+
+USER $NB_USER
+
+COPY create_testenv.sh /tmp/create_testenv.sh
+RUN /bin/bash /tmp/create_testenv.sh --global --no-setup
+
+#  matplotlib nonsense
+ENV XDG_CACHE_HOME /home/$NB_USER/.cache/
+ENV MPLBACKEND=Agg
+# Import matplotlib the first time to build the font cache.
+RUN python -c "import matplotlib.pyplot"
+
+ENV PYTHONPATH $PYTHONPATH:"$HOME"/pymc3

scripts/create_testenv.sh

@@ -2,22 +2,39 @@
 
 set -e # fail on first error
 
+while test $# -gt 0
+do
+    case "$1" in
+        --global)
+            GLOBAL=1
+            ;;
+        --no-setup)
+            NO_SETUP=1
+            ;;
+    esac
+    shift
+done
+
 PYTHON_VERSION=${PYTHON_VERSION:-3.5} # if no python specified, use 3.5
 
-if [[ "$*" != *--global* ]]
+if [ -z ${GLOBAL} ]
 then
-  conda create -n testenv --yes pip python=${PYTHON_VERSION}
-  source activate testenv
+    conda create -n testenv --yes pip python=${PYTHON_VERSION}
+    source activate testenv
 fi
 
 conda install --yes pyqt=4.11.4 jupyter pyzmq numpy scipy nose matplotlib pandas Cython patsy statsmodels joblib coverage
 if [ ${PYTHON_VERSION} == "2.7" ]; then
-  conda install --yes mock enum34;
+    conda install --yes mock enum34;
 fi
 
+pip install --upgrade pip
 pip install tqdm
 pip install --no-deps numdifftools
 pip install git+https://github.com/Theano/Theano.git
 pip install git+https://github.com/mahmoudimus/nose-timer.git
 
-python setup.py build_ext --inplace
+if [ -z ${NO_SETUP} ]
+then
+    python setup.py build_ext --inplace
+fi

scripts/start_container.sh

@@ -0,0 +1,12 @@
+#! /bin/bash
+
+PORT=${PORT:-8888}
+SRC_DIR=${SRC_DIR:-`pwd`}
+NOTEBOOK_DIR=${NOTEBOOK_DIR:-$SRC_DIR/notebooks}
+
+docker build -t pymc3 $SRC_DIR/scripts
+docker run -d \
+    -p $PORT:8888 \
+    -v $SRC_DIR:/home/jovyan/pymc3 \
+    -v $NOTEBOOK_DIR:/home/jovyan/work/ \
+    --name pymc3 pymc3