Merge pull request #386 from joshspeagle/resuming

Resuming of interrupted runs

Author: segasai

CHANGELOG.md

@@ -5,8 +5,13 @@ All notable changes to dynesty will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+*IMPORTANT* This release includes some major refactoring of class structure in dynesty in to implement the check-pointing. While we haven't seen any breakage in our tests, it is possible that some of more-unusual workflows may be affected. Please submit a bug report on github if you see anything that doesn't look correct. Also, while every effort was made that checkpointing works correctly, it is possible that some corner-cases have been missed. Please report if you see any issues.
 
 ### Added
+- The nested samplers can now be saved and restored from the file using .save()
+.restore() interface
+- When sampling is performed using run_nested() it is now possible to perform checkpoints at regular intervals, allowing you then resume sampling if it was interrupted
+
 ### Fixed
 - Sampler.n_effective is no longer unnecessarily computed when sampling with
   an infinite limit on n_effective.

README.md

@@ -1,6 +1,7 @@
 [![Build Status](https://github.com/joshspeagle/dynesty/workflows/Dynesty/badge.svg)](https://github.com/joshspeagle/dynesty/actions)
 [![Documentation Status](https://readthedocs.org/projects/dynesty/badge/?version=latest)](https://dynesty.readthedocs.io/en/latest/?badge=latest)
-[![Coverage Status](https://coveralls.io/repos/github/joshspeagle/dynesty/badge.svg?branch=master)](https://coveralls.io/github/joshspeagle/dynesty?branch=master)
+[![Coverage Status](https://coveralls.io/repos/github/joshspeagle/dynesty/badge.svg?branch=master)](https://coveralls.io/github/joshspeagle/dynesty?branch=master)[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.6609296.svg)](https://doi.org/10.5281/zenodo.6609296)
+
 
 dynesty
 =======

RELEASE.md

@@ -3,7 +3,7 @@ This is an internal memo for things that need to be done for the release
 Essential steps (order is important):
 
 * update the changelog.md and changelog in the docs
-* change the internal version number ( py/dynesty/__init__.py )
+* change the internal version number ( py/dynesty/_version.py )
 * git tag
 * release on pypi and github
 

docs/source/dynamic.rst

@@ -339,7 +339,7 @@ using the :meth:`~dynesty.dynamicsampler.DynamicSampler.run_nested` function::
 
     dsampler.run_nested()
 
-or externally as a generator::
+or externally as a generator (not recommended)::
 
     from dynesty.dynamicsampler import stopping_function, weight_function
 
@@ -442,6 +442,41 @@ could be added to the previous set of samples using::
 
     dsampler.add_batch(nlive=250, maxiter=1000)
 
+Adding more batches
+-------------------
+
+To add more points to the posterior you should be using the :meth:`~dynesty.dynamicsampler.DynamicSampler.add_batch` function. This function has an important parameter that affects how those samples will be generated::
+
+    dsampler.add_batch(mode='auto')
+    dsampler.add_batch(mode='full')
+    dsampler.add_batch(mode='manual', logl_bounds=[-4,1])
+
+The default mode auto will use the weight function described previously to find the best log-likelihood interval to place a batch.
+The full mode will place add a batch that covers the full posterior, i.e. this is equivalent to adding another static nested run to what you have already.
+Finally the manual mode allows you to add a batch that covers a certain specific log-likelihood range.
+
+It is important to understand that there multiple reasons to add batches to a dynamic nested run. One is just to reduce the noise in the posterior/increase the number of effective posterior samples, but another reason is that if you have a very multi-modal problem and you are worried whether you fully sample all the modes, you can do a dynamic run and then keep adding batches till you are satisfied with the result.
+
+
+Checkpointing with the dynamic sampler
+--------------------------------------
+
+Similarly to static nested sampler, the dynamic sampler supports periodic check-pointing into a file if you are sampling using run_nested() interface::
+
+    # initialize our sampler
+    sampler = DynamicNestedSampler(loglike, ptform, ndim, nlive=1000, pool=pool)
+    # run the sampler with checkpointing
+    sampler.run_nested(checkpoint_file='dynesty.save')
+
+And to restore::
+
+    # initialize the sampler
+    sampler = NestedSampler.restore('dynesty.save', pool =mypool)
+    # resume
+    sampler.run_nested(resume=True)
+
+
+    
 Dynamic vs Static
 -----------------
 

docs/source/quickstart.rst

@@ -453,6 +453,7 @@ users an option as to whether they want to parallelize `dynesty` *during*
 runtime (using a user-provided `pool`) or *after* runtime (by merging
 the runs together).
 
+
 Running Internally
 ------------------
 
@@ -520,6 +521,35 @@ to add new samples based on where they left off. This is as easy as::
     sampler.run_nested(dlogz=0.01)
     res3 = sampler.results
 
+Checkpointing
+--------------
+
+While running the sampler using run_nested() interface it is possible to check-point (or save) the state of the sampler into a file at regular intervals. This file can then be used to restart/resume the sampling::
+
+    # initialize our sampler
+    sampler = NestedSampler(loglike, ptform, ndim, nlive=1000)
+    # run the sampler with checkpointing 
+    sampler.run_nested(checkpoint_file='dynesty.save')  
+
+
+You can then restore it now and resume sampling::
+
+    # restore our sampler
+    sampler = NestedSampler.restore('dynesty.save')
+    # resume
+    sampler.run_nested(resume=True)  
+
+
+If you used the pool in the sampler and you want to use the pool after restoring, you need to specify it when restoring::
+
+    mypool = multiprocessing.Pool(6)
+    # restore the sampler
+    sampler = NestedSampler.restore('dynesty.save', pool =mypool)
+    # resume
+    sampler.run_nested(resume=True)
+
+The checkpointing may be helpful if you are running dynesty on HPC with a queue system that has a limit on a wall-time that your jobs can run.
+    
 Running Externally
 ------------------
 

py/dynesty/__init__.py

@@ -10,4 +10,4 @@
 from . import bounding
 from . import utils
 
-__version__ = "1.2.3"
+from ._version import __version__

py/dynesty/_version.py

@@ -0,0 +1 @@
+__version__ = "1.2.3"