📝 Document --runtime_usage and --runtime_buffer

Author: shnizzedy

docs/_sources/developer/nodes.rst

@@ -18,6 +18,8 @@ C-PAC automatically creates a JSON-like file called ``callback.log`` (via the fu
    * specified maximum number of threads per Node, and
    * threads used at runtime.
 
+A ``callback.log`` can be provided to the pipeline configuration file (see :ref:`/user/compute_config`) or with the commandline flag ``--runtime_usage``. If a callback log is provided in the pipeline configuration, nodes with names that match nodes recorded in that pipeline log will have their memory estimates overridden by the values in the callback log plus a buffer percent (provided with the ``--runtime_buffer`` flag or in the pipeline configuration file).
+
 When a developer creates or modifies a Node in C-PAC, a ``mem_gb`` and ``n_procs`` argument should be provided unless the respective defaults of 0.2 and None (number of available system cores) are expected to be sufficient. When testing, the ``mem_gb`` and ``n_procs`` arguments should be adjusted if the observed memory and/or thread usage of a Node exceeds the estimate.
 
 For nodes that will use a varying amount of memory depending on the node's input data, the optional parameter ``mem_x`` takes a tuple of ``(memory multiplier, input file, multiplier mode)`` where ``memory multiplier`` is a number and ``input file`` is the string name of the Node input to multiply such that the memory estimate returned by the ``mem_gb`` attribute is the ``mem_gb`` argument plus ``memory multiplier`` times the dimensions of the ``input file`` as specified in the ``multiplier mode`` (``xyzt`` (spatial × temporal; default), ``xyz`` (spatial), or just ``t`` (temporal)).

docs/_sources/user/compute_config.rst

@@ -8,6 +8,10 @@ Computer Settings
 
 #. **Maximum Cores Per Participant - [integer]:** Number of cores (on a single machine) or slots on a node (cluster/grid) per subject. Slots are cores on a cluster/grid node. 'Number of Cores Per Participant' multiplied by 'Number of Participants to Run Simultaneously' must not be greater than the total number of cores. Dedicating more than one core/CPU per participant will direct C-PAC to parallelize the motion correction and time series registration transform application steps, for a speed increase.
 
+#. **Observed Usage**
+    #. **Callback log - [text]:** The path to a callback log file from a previous run. This file is used override memory estimates with previously observed memory usage. Can be overridden with the commandline flag ``--runtime_usage``.
+    #. **Buffer - [percent]:** A percent of the previously observed memory usage that is to be added to the memory estimate. Default: 10. Can be overridden with the commandline flag ``--runtime_buffer``.
+
 #. **Number of Participants to Run Simultaneously - [integer]:** This number depends on computing resources.
 
 #. **Number of Cores for Anatomical Registration (ANTS) - [integer]:** This number depends on computing resources.