@@ -139,9 +139,11 @@ particular node might use more resources than other nodes in a workflow.
139
139
140
140
SGEGraph
141
141
~~~~~~~~
142
- SGEGraph _ is a exuction plugin working with Sun Grid Engine that allows for
142
+ SGEGraph _ is an execution plugin working with Sun Grid Engine that allows for
143
143
submitting entire graph of dependent jobs at once. This way Nipype does not
144
- need to run a monitoring process - SGE takes care of this.
144
+ need to run a monitoring process - SGE takes care of this. The use of SGEGraph _
145
+ is preferred over SGE _ since the latter adds unnecessary load on the submit
146
+ machine.
145
147
146
148
.. note ::
147
149
@@ -175,6 +177,26 @@ Optional arguments::
175
177
176
178
template: custom template file to use
177
179
sbatch_args: any other command line args to be passed to bsub.
180
+
181
+
182
+ SLURMGraph
183
+ ~~~~~~~~~~
184
+ SLURMGraph _ is an execution plugin working with SLURM that allows for
185
+ submitting entire graph of dependent jobs at once. This way Nipype does not
186
+ need to run a monitoring process - SLURM takes care of this. The use of SLURMGraph _
187
+ plugin is preferred over the vanilla SLURM _ plugin since the latter adds
188
+ unnecessary load on the submit machine.
189
+
190
+
191
+ .. note ::
192
+
193
+ When rerunning unfinished workflows using SLURMGraph you may decide not to
194
+ submit jobs for Nodes that previously finished running. This can speed up
195
+ execution, but new or modified inputs that would previously trigger a Node
196
+ to rerun will be ignored. The following option turns on this functionality::
197
+
198
+ workflow.run(plugin='SLURMGraph', plugin_args = {'dont_resubmit_completed_jobs': True})
199
+
178
200
179
201
HTCondor
180
202
--------
@@ -183,12 +205,12 @@ DAGMan
183
205
~~~~~~
184
206
185
207
With its DAGMan _ component HTCondor _ (previously Condor) allows for submitting
186
- entire graphs of dependent jobs at once (similar to SGEGraph _). With the `` CondorDAGMan `` plug-in
187
- Nipype can utilize this functionality to submit complete workflows directly and
188
- in a single step. Consequently, and in contrast to other plug-ins, workflow
189
- execution returns almost instantaneously -- Nipype is only used to generate the
190
- workflow graph, while job scheduling and dependency resolution are entirely
191
- managed by HTCondor _.
208
+ entire graphs of dependent jobs at once (similar to SGEGraph _ and SLURMGaaoh _ ).
209
+ With the `` CondorDAGMan `` plug-in Nipype can utilize this functionality to
210
+ submit complete workflows directly and in a single step. Consequently, and
211
+ in contrast to other plug-ins, workflow execution returns almost
212
+ instantaneously -- Nipype is only used to generate the workflow graph,
213
+ while job scheduling and dependency resolution are entirely managed by HTCondor _.
192
214
193
215
Please note that although DAGMan _ supports specification of data dependencies
194
216
as well as data provisioning on compute nodes this functionality is currently
0 commit comments