Skip to content

Commit 99673f4

Browse files
shnizzedysgiavasis
andcommitted
🔊 Add lens of strat_list_list and rp_dct to engine.log
> I found a couple really good spots where we can add reports for verbose > logging/debugging, to help quickly figure out these forking issues in > the future. Check the length of strat_list_list right before this loop: > https://github.com/FCP-INDI/C-PAC/blob/a33dfe892ce3cd31d346f3c1ab032e7e91e4ba0f/CPAC/pipeline/engine.py#L436, > and you get something like this: > > ``` > 220203-07:26:12,186 nipype.workflow INFO: > Connecting n4_bias_correction... > 220203-07:26:12,187 nipype.workflow INFO: > Connecting acpc_alignment_head_with_mask... > 1 > 220203-07:26:12,197 nipype.workflow INFO: > Connecting freesurfer_abcd_preproc... > 1 > 220203-07:26:12,214 nipype.workflow INFO: > Connecting brain_mask_freesurfer_abcd... > 2 > 220203-07:26:12,224 nipype.workflow INFO: > Connecting brain_extraction... > 3 > 220203-07:26:12,230 nipype.workflow INFO: > Connecting register_ANTs_anat_to_template... > 9 > 220203-07:26:12,712 nipype.workflow INFO: > Connecting overwrite_transform_anat_to_template... > ``` > so you see that brain_mask_freesurfer_abcd suddenly has two forks, and > it starts multiplying out unnecessarily after that- and blows up to 9 > right before moving onto the first node block that takes forever > (overwrite_transform_anat_to_template) - if you let that progress I'm > sure you'll see some crazy number way higher than 9. > but that wasn't good enough to find out what the problem was (at first > I assumed wmparc and T1w had to be linked but that didn't do it) - so I > started dropping the amount of strats for each specific resource pulled > into a node block - if you report the resource, and the `len(rp_dct)` > of each resource [here](https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/pipeline/engine.py#L396), > you quickly see that `freesurfer-subject-dir` is the only one with > multiple strats/forks in freesurfer_abcd_preproc, and tracing that > resource's provenance led me to see that there were two versions of it, > one from regular `freesurfer_preproc` and the other from > `freesurfer_abcd_preproc` Co-authored-by: Steve Giavasis <[email protected]>
1 parent 3308d29 commit 99673f4

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

CPAC/pipeline/engine.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -404,7 +404,8 @@ def get_strats(self, resources, debug=False):
404404
len_inputs -= 1
405405
continue
406406
sub_pool = []
407-
407+
if debug:
408+
verbose_logger.debug('len(rp_dct): %s\n', len(rp_dct))
408409
for strat in rp_dct.keys():
409410
json_info = self.get_json(fetched_resource, strat)
410411
cpac_prov = json_info['CpacProvenance']
@@ -447,6 +448,9 @@ def get_strats(self, resources, debug=False):
447448
strat_str_list.append(strat_str)
448449
strat_list_list.append(strat_list)
449450

451+
if debug:
452+
verbose_logger.debug('len(strat_list_list): %s\n',
453+
len(strat_list_list))
450454
for strat_list in strat_list_list:
451455

452456
json_dct = {}

0 commit comments

Comments
 (0)