📝 Document "✨ Set constant random seed via config or command line" (FCP-INDI/C-PAC#1646) (#271)

Author: shnizzedy

docs/_sources/developer/index.rst

@@ -16,6 +16,7 @@ Contents:
    installation
    pipeline
    nodes
+   random_state
    workflow_documentation
    workflows/index
    testing

docs/_sources/developer/nodes.rst

@@ -1,7 +1,6 @@
 *****
 Nodes
 *****
-
 .. _mem_gb:
 
 .. _n_procs:
@@ -23,8 +22,10 @@ When a developer creates or modifies a Node in C-PAC, a ``mem_gb`` and ``n_procs
 
 For nodes that will use a varying amount of memory depending on the node's input data, the optional parameter ``mem_x`` takes a tuple of ``(memory multiplier, input file, multiplier mode)`` where ``memory multiplier`` is a number and ``input file`` is the string name of the Node input to multiply such that the memory estimate returned by the ``mem_gb`` attribute is the ``mem_gb`` argument plus ``memory multiplier`` times the dimensions of the ``input file`` as specified in the ``multiplier mode`` (``xyzt`` (spatial × temporal; default), ``xyz`` (spatial), or just ``t`` (temporal)).
 
-.. note::
-   ``mem_x`` is a new parameter in C-PAC v1.8.1 and subject to change in future releases as we continue to `develop methods for setting data- and operation-dependent memory estimates <https://github.com/FCP-INDI/C-PAC/issues/1509>`_.
+
+   .. note::
+      ``mem_x`` is a new parameter in C-PAC v1.8.1 and subject to change in future releases as we continue to `develop methods for setting data- and operation-dependent memory estimates <https://github.com/FCP-INDI/C-PAC/issues/1509>`_.
+
 
 .. autoclass:: CPAC.pipeline.nipype_pipeline_engine.Node
    :special-members: __init__
@@ -36,17 +37,33 @@ For nodes that will use a varying amount of memory depending on the node's input
    :members:
    :inherited-members:
 
+.. autoclass:: nipype.interfaces.base.core.Interface
+   :special-members: __init__
+   :members:
+   :inherited-members:
+
+
+   .. seealso::
+      `Nipype: Interface Specifications <https://nipype.readthedocs.io/en/1.5.1/devel/interface_specs.html>`_
+
+
+.. autoclass:: CPAC.utils.interfaces.function.Function
+   :special-members: __init__
+   :members:
+   :inherited-members:
+
 .. autoclass:: CPAC.pipeline.nipype_pipeline_engine.Workflow
    :special-members: __init__
    :members:
    :inherited-members:
 
+
 The Nipype utility function :py:func:`~nipype.utils.draw_gantt_chart.log_to_dict` reads a log file generated by ``log_nodes_cb`` to a Python dictionary.
 
+
 .. autofunction:: CPAC.utils.monitoring.log_nodes_cb
 
-draw_gantt_chart
-================
+
 
 .. automodule:: CPAC.utils.monitoring.draw_gantt_chart
    :members: resource_report, log_to_dict, generate_gantt_chart, resource_overusage_report

docs/_sources/developer/random_state.rst

@@ -0,0 +1,13 @@
+Random State
+============
+
+When performing reproducibility/variability experiments, it is important to isolate sources of variability. One such source is random state.
+
+Users can fix the random state of C-PAC's execution by specifying a random seed. When adding a Node that should accept a user-specified random seed, add
+
+* the :py:class:`nipype.interfaces.base.core.Interface` and the corresponding flags to add/remove to set the random seed, or
+* in the case of a :py:class:`CPAC.utils.interfaces.function.Function` Node, the Function node's function and a function to apply to that function definition to set the random seed
+
+to :py:func:`CPAC.pipeline.random_state.seed.random_seed_flags`.
+
+.. autofunction:: CPAC.pipeline.random_state.seed.random_seed_flags

docs/_sources/user/pipelines/design_a_pipeline.rst

@@ -13,21 +13,21 @@ Design A Pipeline
 
 C-PAC offers a graphical interface you can use to quickly and easily modify the default pipeline or create your own from scratch: `https://fcp-indi.github.io/C-PAC_GUI/ <https://fcp-indi.github.io/C-PAC_GUI/>`_
 
-.. note::
+    .. note::
+
+    Currently the GUI creates a C-PAC v1.6.0 pipeline configuration file. This syntax persisted through v1.7.2 but is deprecated with the release of v1.8.0.
 
-   Currently the GUI creates a C-PAC v1.6.0 pipeline configuration file. This syntax persisted through v1.7.2 but is deprecated with the release of v1.8.0.
+    If given a pipeline file in the older syntax, C-PAC v1.8 will attempt to convert the pipeline configuration file to the new syntax, saving the converted file in your output directory.
 
-   If given a pipeline file in the older syntax, C-PAC v1.8 will attempt to convert the pipeline configuration file to the new syntax, saving the converted file in your output directory.
+    An update to the GUI to create v1.8.0 syntax configuration files is underway.
 
-   An update to the GUI to create v1.8.0 syntax configuration files is underway.
+    The newer (v1.8) syntax will not work with older versions of C-PAC.
 
-   The newer (v1.8) syntax will not work with older versions of C-PAC.
+    See :ref:`using_a_text_editor` for configuring a custom pipeline without the GUI.
+    
+.. seealso::
 
-   See :ref:`using_a_text_editor` for configuring a custom pipeline without the GUI.
-   
-   .. seealso::
-   
-      :doc:`Details about mapping the older syntax to the new. </user/pipelines/1.7-1.8-nesting-mappings>`
+    :doc:`Details about mapping the older syntax to the new. </user/pipelines/1.7-1.8-nesting-mappings>`
 
 .. figure:: /_images/gui_home1.png
 

docs/_sources/user/pipelines/pipeline_config.rst

@@ -37,15 +37,10 @@ Definitions
     .. include:: /glossary/template.rst
         :start-line: 3
 
-.. rubric:: Reference
-
-.. bibliography:: /references/glossary.bib
-   :style: cpac_docs_style
-   :cited:
-   :keyprefix: glossary-
 
 .. include:: design_a_pipeline.rst
 
+
 .. _using_a_text_editor:
 
 Using a Text Editor
@@ -109,6 +104,7 @@ Data Management and Environment Settings
 
 * :doc:`Computer Settings </user/compute_config>`
 * :doc:`Output Settings </user/output_config>`
+* :doc:`Random State </user/pipelines/random_state>`
 
 Pre- and post-processing
 '''''''''''''''''''''''''
@@ -127,3 +123,12 @@ Derivatives
 * :doc:`Amplitude of Low Frequency Fluctuations (ALFF) and fractional ALFF (fALFF) </user/alff>` - Measure the power of slow fluctuations in brain activity.
 * :doc:`Regional Homogeneity (ReHo) </user/reho>` - Measure the similarity of activity patterns across neighboring voxels.
 * :doc:`Network Centrality </user/centrality>` - Analyze the structure of functional networks.
+
+
+.. rubric:: Reference
+
+
+.. bibliography:: /references/glossary.bib
+    :style: cpac_docs_style
+    :cited:
+    :keyprefix: glossary-

docs/_sources/user/pipelines/random_state.rst

@@ -0,0 +1,34 @@
+Random State
+============
+
+When performing reproducibility/variability experiments, it is important to isolate sources of variability. One such source is random state.
+
+Random seed can be set in pipeline config
+
+.. literalinclude:: /references/default_pipeline.yml
+   :language: YAML
+   :start-at: pipeline_setup:
+   :lines: 1
+
+.. literalinclude:: /references/default_pipeline.yml
+   :language: YAML
+   :start-at: system_config:
+   :end-before: # Select Off if you intend to run CPAC on a single machine.
+
+or on the command line with ``--random_seed $SEED``.
+
+Valid options are positive integers up to 2,147,483,647 or the word 'random' (which will set an integer in that range). If not specified, a seed will not be set, and each relevant process will run with an undocumented random seed.
+
+When a seed is set, a ``random.log`` file, including the constant seed and each node the seed was applied to, will be generated in the logging directory.
+
+The following processes currently support this feature:
+
+.. exec::
+    from CPAC.pipeline.random_state.seed import random_seed_flags, \
+                                                set_up_random_state
+    set_up_random_state('random')
+    processes = random_seed_flags()
+    for interface in processes['interfaces'].keys():
+        print(interface._cmd)
+    for fxn in processes['functions'].keys():
+        print('.'.join([fxn.__module__, fxn.__name__]))

requirements.txt

@@ -1,4 +1,5 @@
 m2r
+mistune==0.8.4
 numpydoc
 PyGithub
 sphinx