DOC: fix data package pages for doctests

Make doctests pass by disabling some tests, fixing others, and making a
nipy test data directory framework on the fly to test against.

Some typos and bad formatting.  Reflow of paragraphs with 78 char width.

Author: matthew-brett

doc/source/devel/data_pkg_design.rst

@@ -3,24 +3,42 @@
 Design of data packages for the nipy suite
 ==========================================
 
-See :ref:`data-package-discuss` for a more general discussion of design issues.
+See :ref:`data-package-discuss` for a more general discussion of design
+issues.
 
-When developing or using nipy, many data files can be useful. We divide
-the data files nipy uses into at least 3 categories
+When developing or using nipy, many data files can be useful. We divide the
+data files nipy uses into at least 3 categories
 
 #. *test data* - data files required for routine code testing
 #. *template data* - data files required for algorithms to function,
    such as templates or atlases
 #. *example data* - data files for running examples, or optional tests
 
 Files used for routine testing are typically very small data files. They are
-shipped with the software, and live in the code repository. For example, in the
-case of ``nipy`` itself, there are some test files that live in the module path
-``nipy.testing.data``.
+shipped with the software, and live in the code repository. For example, in
+the case of ``nipy`` itself, there are some test files that live in the module
+path ``nipy.testing.data``.
 
 *template data* and *example data* are example of *data packages*.  What
 follows is a discussion of the design and use of data packages.
 
+.. testsetup::
+
+    # Make fake data and template directories
+    import os
+    from os.path import join as pjoin
+    import tempfile
+    tmpdir = tempfile.mkdtemp()
+    os.environ['NIPY_USER_DIR'] = tmpdir
+    for subdir in ('data', 'templates'):
+        files_dir = pjoin(tmpdir, 'nipy', subdir)
+        os.makedirs(files_dir)
+        with open(pjoin(files_dir, 'config.ini'), 'wt') as fobj:
+            fobj.write(
+    """[DEFAULT]
+    version = 0.2
+    """)
+
 Use cases for data packages
 +++++++++++++++++++++++++++
 
@@ -33,28 +51,29 @@ The programmer will want to use the data something like this:
 
    from nibabel.data import make_datasource
 
-   templates = make_datasource('nipy', 'templates')
+   templates = make_datasource(dict(relpath='nipy/templates'))
    fname = templates.get_filename('ICBM152', '2mm', 'T1.nii.gz')
-   
+
 where ``fname`` will be the absolute path to the template image
-``ICBM152/2mm/T1.nii.gz``. 
+``ICBM152/2mm/T1.nii.gz``.
 
 The programmer can insist on a particular version of a ``datasource``:
 
-.. testcode::
-
-   if templates.version < '0.4':
-      raise ValueError('Need datasource version at least 0.4')
+>>> if templates.version < '0.4':
+...     raise ValueError('Need datasource version at least 0.4')
+Traceback (most recent call last):
+...
+ValueError: Need datasource version at least 0.4
 
 If the repository cannot find the data, then:
 
->>> make_datasource('nipy', 'implausible')
-Traceback
+>>> make_datasource(dict(relpath='nipy/implausible'))
+Traceback (most recent call last):
  ...
-nibabel.data.DataError
+nibabel.data.DataError: ...
 
 where ``DataError`` gives a helpful warning about why the data was not
-found, and how it should be installed.  
+found, and how it should be installed.
 
 Warnings during installation
 ````````````````````````````
@@ -68,8 +87,8 @@ data when installing the package.  Thus::
 will import nipy after installation to check whether these raise an error:
 
 >>> from nibabel.data import make_datasource
->>> template = make_datasource('nipy', 'templates')
->>> example_data = make_datasource('nipy', 'data')
+>>> templates = make_datasource(dict(relpath='nipy/templates'))
+>>> example_data = make_datasource(dict(relpath='nipy/data'))
 
 and warn the user accordingly, with some basic instructions for how to
 install the data.
@@ -82,7 +101,7 @@ Finding the data
 The routine ``make_datasource`` will need to be able to find the data
 that has been installed.  For the following call:
 
->>> templates = make_datasource('nipy', 'templates')
+>>> templates = make_datasource(dict(relpath='nipy/templates'))
 
 We propose to:
 
@@ -130,8 +149,8 @@ umbrella of neuroimaging in python - and the NIPY package - the main
 code package in the NIPY project.  Thus, if you want to install data
 under the NIPY *package* umbrella, your data might go to
 ``/usr/share/nipy/nipy/packagename`` (on Unix).  Note ``nipy`` twice -
-once for the project, once for the pacakge.  If you want to install data
-under - say - the ```pbrain`` package umbrella, that would go in
+once for the project, once for the package.  If you want to install data
+under - say - the ``pbrain`` package umbrella, that would go in
 ``/usr/share/nipy/pbrain/packagename``.
 
 Data package format
@@ -141,7 +160,7 @@ The following tree is an example of the kind of pattern we would expect
 in a data directory, where the ``nipy-data`` and ``nipy-templates``
 packages have been installed::
 
-  <ROOT> 
+  <ROOT>
   `-- nipy
       |-- data
       |   |-- config.ini
@@ -191,8 +210,9 @@ For the example above this will result in these subdirectories::
 because ``nipy`` is both the project, and the package to which the data
 relates.
 
-If you install to a particular location, you will need to add that
-location to the output of ``nipy.data.get_data_path()`` using one of the mechanisms above, for example, in your system configuration::
+If you install to a particular location, you will need to add that location to
+the output of ``nipy.data.get_data_path()`` using one of the mechanisms above,
+for example, in your system configuration::
 
    export NIPY_DATA_PATH=/my/prefix/share/nipy
 
@@ -202,13 +222,13 @@ Packaging for distributions
 For a particular data package - say ``nipy-templates`` - distributions
 will want to:
 
-#. Install the data in set location.  The default from ``python setup.py install`` for the data packages will be ``/usr/share/nipy`` on Unix.
-#. Point a system installation of NIPY to these data. 
+#. Install the data in set location.  The default from ``python setup.py
+   install`` for the data packages will be ``/usr/share/nipy`` on Unix.
+#. Point a system installation of NIPY to these data.
 
-For the latter, the most obvious route is to copy an ``.ini`` file named
-for the data package into the NIPY ``etc_dir``.  In this case, on Unix,
-we will want a file called ``/etc/nipy/nipy_templates.ini`` with
-contents::
+For the latter, the most obvious route is to copy an ``.ini`` file named for
+the data package into the NIPY ``etc_dir``.  In this case, on Unix, we will
+want a file called ``/etc/nipy/nipy_templates.ini`` with contents::
 
    [DATA]
    path = /usr/share/nipy
@@ -219,16 +239,17 @@ Current implementation
 This section describes how we (the nipy community) implement data packages at
 the moment.
 
-The data in the data packages will not usually be under source control.  This is
-because images don't compress very well, and any change in the data will result
-in a large extra storage cost in the repository.  If you're pretty clear that
-the data files aren't going to change, then a repository could work OK.
+The data in the data packages will not usually be under source control.  This
+is because images don't compress very well, and any change in the data will
+result in a large extra storage cost in the repository.  If you're pretty
+clear that the data files aren't going to change, then a repository could work
+OK.
 
-The data packages will be available at a central release location.  For
-now this will be: http://nipy.org/data-packages/ .
+The data packages will be available at a central release location.  For now
+this will be: http://nipy.org/data-packages/ .
 
-A package, such as ``nipy-templates-0.2.tar.gz`` will have the following
-sort of structure::
+A package, such as ``nipy-templates-0.2.tar.gz`` will have the following sort
+of structure::
 
 
   <ROOT>
@@ -248,26 +269,31 @@ sort of structure::
 
 
 There should be only one ``nipy/packagename`` directory delivered by a
-particular package.  For example, this package installs
-``nipy/templates``, but does not contain ``nipy/data``.
+particular package.  For example, this package installs ``nipy/templates``,
+but does not contain ``nipy/data``.
 
 Making a new package tarball is simply:
 
-#. Downloading and unpacking e.g ``nipy-templates-0.1.tar.gz`` to form
-   the directory structure above.
-#. Making any changes to the directory
+#. Downloading and unpacking e.g. ``nipy-templates-0.1.tar.gz`` to form the
+   directory structure above;
+#. Making any changes to the directory;
 #. Running ``setup.py sdist`` to recreate the package.
 
 The process of making a release should be:
 
-#. Increment the major or minor version number in the ``config.ini`` file
-#. Make a package tarball as above
-#. Upload to distribution site
+#. Increment the major or minor version number in the ``config.ini`` file;
+#. Make a package tarball as above;
+#. Upload to distribution site.
 
 There is an example nipy data package ``nipy-examplepkg`` in the
 ``examples`` directory of the NIPY repository.
 
 The machinery for creating and maintaining data packages is available at
-http://github.com/nipy/data-packaging
+http://github.com/nipy/data-packaging.
 
 See the ``README.txt`` file there for more information.
+
+.. testcleanup::
+
+    import shutil
+    shutil.rmtree(tmpdir)

doc/source/devel/data_pkg_uses.rst

@@ -26,44 +26,46 @@ We'll call our package `dang` - data package new generation.
 Create local-path prundle
 -------------------------
 
->>> import os
->>> import tempfile
->>> pth = tempfile.mkdtemp() # temporary directory
+::
 
-Make a pinstance object
+    >>> import os
+    >>> import tempfile
+    >>> pth = tempfile.mkdtemp() # temporary directory
 
->>> from dang import Pinstance
->>> pri = Prundle(name='my-package')
->>> pri.pkg_name
-'my-package'
->>> pri.meta
-{}
+Make a pinstance object::
 
-Now we make a prundle.   First a directory to contain it
+    >>> from dang import Pinstance
+    >>> pri = Prundle(name='my-package')
+    >>> pri.pkg_name
+    'my-package'
+    >>> pri.meta
+    {}
+
+Now we make a prundle.   First a directory to contain it::
 
->>> import os
->>> import tempfile
->>> pth = tempfile.mkdtemp() # temporary directory
+    >>> import os
+    >>> import tempfile
+    >>> pth = tempfile.mkdtemp() # temporary directory
 
->>> from dang.prundle import LocalPathPrundle
->>> prun = LocalPathPrundle(pri, pth)
+    >>> from dang.prundle import LocalPathPrundle
+    >>> prun = LocalPathPrundle(pri, pth)
 
 At the moment there's nothing in the directory.  The 'write' method will write
-the meta information - here just the package name.
+the meta information - here just the package name::
 
->>> prun.write() # writes meta.ini file
->>> os.listdir(pth)
-['meta.ini']
+    >>> prun.write() # writes meta.ini file
+    >>> os.listdir(pth)
+    ['meta.ini']
 
 The local path prundle data is just the set of files in the temporary directory
 named in ``pth`` above.
 
 Now we've written the package, we can get it by a single call that reads in the
-``meta.ini`` file:
+``meta.ini`` file::
 
->>> prun_back = LocalPathPrundle.from_path(pth)
->>> prun_back.pkg_name
-'my-package'
+    >>> prun_back = LocalPathPrundle.from_path(pth)
+    >>> prun_back.pkg_name
+    'my-package'
 
 Getting prundle data
 --------------------
@@ -79,8 +81,9 @@ In fact, local path distribution objects also have a ``path`` attribute::
 
     >>> fname = os.path.join(prun.path, 'a_file.txt')
 
-The ``path`` attribute might not make sense for objects with greater abstraction
-over the file-system - for example objects encapsulating web content.
+The ``path`` attribute might not make sense for objects with greater
+abstraction over the file-system - for example objects encapsulating web
+content.
 
 *********
 Discovery
@@ -93,8 +96,8 @@ We want to be able to tell the system where prundles are - and the system will
 then be able to return a prundle on request - perhaps by package name.  The
 system here is answering a :ref:`prundle-discovery` query.
 
-We will then want to ask our packaging system whether it knows about the prundle
-we are interested in.
+We will then want to ask our packaging system whether it knows about the
+prundle we are interested in.
 
 Discovery sources
 =================
@@ -150,8 +153,8 @@ from a list of sources.  Something like this::
     >>> dq_res[0].pkg_name
     'my-package'
 
-We'll often want to do exactly this, so we'll add this source pool to those that
-can be returned from our ``get_source`` convenience function::
+We'll often want to do exactly this, so we'll add this source pool to those
+that can be returned from our ``get_source`` convenience function::
 
     >>> src_pool = dpkg.get_source('local-pool')
 
@@ -202,7 +205,7 @@ Discovery sources
 =================
 
 The discovery source has to be able to return prundle objects for the
-prundles it knows about.
+prundles it knows about::
 
     [my-package]
     0 = /some/path
@@ -213,19 +216,19 @@ prundles it knows about.
 Registering a package
 =====================
 
-So far we have a local path distribution, that is a directory with some files in
-it, and our own ``meta.ini`` file, containing the package name and version.  How
-does this package register itself to the default sources?  Of course, we could
-use ``dpkg`` as above::
+So far we have a local path distribution, that is a directory with some files
+in it, and our own ``meta.ini`` file, containing the package name and version.
+How does this package register itself to the default sources?  Of course, we
+could use ``dpkg`` as above::
 
     >>> dst = dpkg.LocalPathDistribution.from_path(path='/a/path')
     >>> local_usr = dpkg.get_source('local-user')
     >>> local_usr.register(dst)
     >>> local_usr.save()
 
-but we wanted to be able to avoid using ``dpkg``.  To do this, there might be a
-supporting script, in the distribution directory, called ``register_me.py``, of
-form given in :download:`register_me.py`.
+but we wanted to be able to avoid using ``dpkg``.  To do this, there might be
+a supporting script, in the distribution directory, called ``register_me.py``,
+of form given in :download:`register_me.py`.
 
 Using discovery sources without dpkg
 ====================================