Skip to content

Commit 9c83152

Browse files
committed
Improved docs.
1 parent 5fb4b94 commit 9c83152

File tree

1 file changed

+30
-8
lines changed

1 file changed

+30
-8
lines changed

doc/users/plugins.rst

Lines changed: 30 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -139,9 +139,11 @@ particular node might use more resources than other nodes in a workflow.
139139
140140
SGEGraph
141141
~~~~~~~~
142-
SGEGraph_ is a exuction plugin working with Sun Grid Engine that allows for
142+
SGEGraph_ is an execution plugin working with Sun Grid Engine that allows for
143143
submitting entire graph of dependent jobs at once. This way Nipype does not
144-
need to run a monitoring process - SGE takes care of this.
144+
need to run a monitoring process - SGE takes care of this. The use of SGEGraph_
145+
is preferred over SGE_ since the latter adds unnecessary load on the submit
146+
machine.
145147

146148
.. note::
147149

@@ -175,6 +177,26 @@ Optional arguments::
175177

176178
template: custom template file to use
177179
sbatch_args: any other command line args to be passed to bsub.
180+
181+
182+
SLURMGraph
183+
~~~~~~~~~~
184+
SLURMGraph_ is an execution plugin working with SLURM that allows for
185+
submitting entire graph of dependent jobs at once. This way Nipype does not
186+
need to run a monitoring process - SLURM takes care of this. The use of SLURMGraph_
187+
plugin is preferred over the vanilla SLURM_ plugin since the latter adds
188+
unnecessary load on the submit machine.
189+
190+
191+
.. note::
192+
193+
When rerunning unfinished workflows using SLURMGraph you may decide not to
194+
submit jobs for Nodes that previously finished running. This can speed up
195+
execution, but new or modified inputs that would previously trigger a Node
196+
to rerun will be ignored. The following option turns on this functionality::
197+
198+
workflow.run(plugin='SLURMGraph', plugin_args = {'dont_resubmit_completed_jobs': True})
199+
178200

179201
HTCondor
180202
--------
@@ -183,12 +205,12 @@ DAGMan
183205
~~~~~~
184206

185207
With its DAGMan_ component HTCondor_ (previously Condor) allows for submitting
186-
entire graphs of dependent jobs at once (similar to SGEGraph_). With the ``CondorDAGMan`` plug-in
187-
Nipype can utilize this functionality to submit complete workflows directly and
188-
in a single step. Consequently, and in contrast to other plug-ins, workflow
189-
execution returns almost instantaneously -- Nipype is only used to generate the
190-
workflow graph, while job scheduling and dependency resolution are entirely
191-
managed by HTCondor_.
208+
entire graphs of dependent jobs at once (similar to SGEGraph_ and SLURMGaaoh_).
209+
With the ``CondorDAGMan`` plug-in Nipype can utilize this functionality to
210+
submit complete workflows directly and in a single step. Consequently, and
211+
in contrast to other plug-ins, workflow execution returns almost
212+
instantaneously -- Nipype is only used to generate the workflow graph,
213+
while job scheduling and dependency resolution are entirely managed by HTCondor_.
192214

193215
Please note that although DAGMan_ supports specification of data dependencies
194216
as well as data provisioning on compute nodes this functionality is currently

0 commit comments

Comments
 (0)