-
-
Notifications
You must be signed in to change notification settings - Fork 41
Open
Description
Describe the bug
241023-08:14:13,673 nipype.workflow INFO:
Connecting warp_longitudinal_T1w_to_template...
2024-10-23 08:14:13,673 [ INFO] Connecting warp_longitudinal_T1w_to_template...
241023-08:14:13,674 nipype.workflow INFO:
Connecting warp_longitudinal_T1w_to_template...
2024-10-23 08:14:13,674 [ INFO] Connecting warp_longitudinal_T1w_to_template...
Traceback (most recent call last):
File "/code/run.py", line 833, in <module>
raise exception
File "/code/run.py", line 827, in <module>
run_main()
File "/code/run.py", line 796, in run_main
exitcode = CPAC.pipeline.cpac_runner.run(
File "/code/CPAC/pipeline/cpac_runner.py", line 425, in run
run_T1w_longitudinal(sublist, c)
File "/code/CPAC/pipeline/cpac_runner.py", line 209, in run_T1w_longitudinal
anat_longitudinal_wf(subject_id, sub_list, cfg)
File "/code/CPAC/longitudinal_pipeline/longitudinal_workflow.py", line 659, in anat_longitudinal_wf
wf = connect_pipeline(wf, config, rpool, pipeline_blocks)
File "/code/CPAC/pipeline/cpac_pipeline.py", line 1127, in connect_pipeline
wf = nb.connect_block(wf, cfg, rpool)
File "/code/CPAC/pipeline/engine.py", line 1499, in connect_block
for pipe_idx, strat_pool in rpool.get_strats(
File "/code/CPAC/pipeline/engine.py", line 558, in get_strats
raise LookupError('\n\n[!] C-PAC says: None of the listed '
LookupError: When trying to connect node block 'warp_longitudinal_T1w_to_template' to workflow 'cpac_sub-70009_ses-2' at beginning:
[!] C-PAC says: None of the listed resources in the node block being connected exist in the resource pool.
Resources:
[]
To reproduce
Try to run a longitudinal pipeline
Custom pipeline configuration
pipeline_config_rbc-options_addRegressors_noBBR_bugfix_funcmask_FSL_long.yml
%YAML 1.1
---
# CPAC Pipeline Configuration YAML file
# Version 1.8.7.dev1
#
# http://fcp-indi.github.io for more info.
#
# Tip: This file can be edited manually with a text editor for quick modifications.
FROM: fmriprep-options
pipeline_setup:
# Name for this pipeline configuration - useful for identification.
# This string will be sanitized and used in filepaths
pipeline_name: RBC_addRegressors_noBBR_bugfixBPF_funcmask_FSL
output_directory:
# Quality control outputs
quality_control:
# Generate eXtensible Connectivity Pipeline-style quality control files
generate_xcpqc_files: On
# Generate quality control pages containing preprocessing and derivative outputs.
generate_quality_control_images: Off
system_config:
# Stop worklow execution on first crash?
fail_fast: On
# Random seed used to fix the state of execution.
# If unset, each process uses its own default.
# If set, a `random.log` file will be generated logging the random seed and each node to which that seed was applied.
# If set to a positive integer (up to 2147483647), that integer will be used to seed each process that accepts a random seed.
# If set to 'random', a random positive integer (up to 2147483647) will be generated and that seed will be used to seed each process that accepts a random seed.
random_seed: 77742777
longitudinal_template_generation:
# If you have multiple T1w's, you can generate your own run-specific custom
# T1w template to serve as an intermediate to the standard template for
# anatomical registration.
# This runs before the main pipeline as it requires multiple T1w sessions
# at once.
run: On
# Freesurfer longitudinal template algorithm using FSL FLIRT
# Method to average the dataset at each iteration of the template creation
# Options: median, mean or std
average_method: median
# Degree of freedom for FLIRT in the template creation
# Options: 12 (affine), 9 (traditional), 7 (global rescale) or 6 (rigid body)
dof: 12
# Interpolation parameter for FLIRT in the template creation
# Options: trilinear, nearestneighbour, sinc or spline
interp: trilinear
# Cost function for FLIRT in the template creation
# Options: corratio, mutualinfo, normmi, normcorr, leastsq, labeldiff or bbr
cost: corratio
# Number of threads used for one run of the template generation algorithm
thread_pool: 2
# Threshold of transformation distance to consider that the loop converged
# (-1 means numpy.finfo(np.float64).eps and is the default)
convergence_threshold: -1
registration_workflows:
anatomical_registration:
# Template to be used during registration.
# It is not necessary to change this path unless you intend to use a non-standard template.
T1w_brain_template: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain.nii.gz
# Template to be used during registration.
# It is not necessary to change this path unless you intend to use a non-standard template.
T1w_template: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}.nii.gz
# Template to be used during registration.
# It is not necessary to change this path unless you intend to use a non-standard template.
T1w_brain_template_mask: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz
functional_registration:
coregistration:
# functional (BOLD/EPI) registration to anatomical (structural/T1)
run: On
func_input_prep:
# Choose whether to use the mean of the functional/EPI as the input to functional-to-anatomical registration or one of the volumes from the functional 4D timeseries that you choose.
# input: ['Mean_Functional', 'Selected_Functional_Volume', 'fmriprep_reference']
input: [Mean_Functional] #changed from fmriprep_reference
boundary_based_registration:
# this is a fork point
# run: [On, Off] - this will run both and fork the pipeline
run: [Off]
# reference for boundary based registration
# options: 'whole-head' or 'brain'
reference: brain
# choose which FAST map to generate BBR WM mask
# options: 'probability_map', 'partial_volume_map'
bbr_wm_map: partial_volume_map
# optional FAST arguments to generate BBR WM mask
bbr_wm_mask_args: -bin
func_registration_to_template:
output_resolution:
# The resolution (in mm) to which the preprocessed, registered functional timeseries outputs are written into.
# NOTE:
# selecting a 1 mm or 2 mm resolution might substantially increase your RAM needs- these resolutions should be selected with caution.
# for most cases, 3 mm or 4 mm resolutions are suggested.
# NOTE:
# this also includes the single-volume 3D preprocessed functional data,
# such as the mean functional (mean EPI) in template space
func_preproc_outputs: 2mm
# The resolution (in mm) to which the registered derivative outputs are written into.
# NOTE:
# this is for the single-volume functional-space outputs (i.e. derivatives)
# thus, a higher resolution may not result in a large increase in RAM needs as above
func_derivative_outputs: 2mm
target_template:
T1_template:
# Standard Skull Stripped Template. Used as a reference image for functional registration.
# This can be different than the template used as the reference/fixed for T1-to-template registration.
T1w_brain_template_funcreg: $FSLDIR/data/standard/MNI152_T1_${func_resolution}_brain.nii.gz
# Standard Anatomical Brain Image with Skull.
# This can be different than the template used as the reference/fixed for T1-to-template registration.
T1w_template_funcreg: $FSLDIR/data/standard/MNI152_T1_${func_resolution}.nii.gz
# Template to be used during registration.
# It is not necessary to change this path unless you intend to use a non-standard template.
T1w_brain_template_mask_funcreg: $FSLDIR/data/standard/MNI152_T1_${func_resolution}_brain_mask.nii.gz
# a standard template for resampling if using float resolution
T1w_template_for_resample: $FSLDIR/data/standard/MNI152_T1_${func_resolution}_brain.nii.gz
functional_preproc:
# distortion_correction:
# using: ['PhaseDiff', 'Blip', 'Blip-FSL-TOPUP']
# PhaseDiff - Perform field map correction using a single phase difference image, a subtraction of the two phase images from each echo. Default scanner for this method is SIEMENS.
# Blip - Uses AFNI 3dQWarp to calculate the distortion unwarp for EPI field maps of opposite/same phase encoding direction.
# Blip-FSL-TOPUP - Uses FSL TOPUP to calculate the distortion unwarp for EPI field maps of opposite/same phase encoding direction.
# using: [PhaseDiff, Blip-FSL-TOPUP]
func_masking:
run: On
# using: ['AFNI', 'FSL', 'FSL_AFNI', 'Anatomical_Refined', 'Anatomical_Based', 'Anatomical_Resampled', 'CCS_Anatomical_Refined']
# FSL_AFNI: fMRIPrep-style BOLD mask. Ref: https://github.com/nipreps/niworkflows/blob/a221f612/niworkflows/func/util.py#L246-L514
# Anatomical_Refined: 1. binarize anat mask, in case it is not a binary mask. 2. fill holes of anat mask 3. init_bold_mask : input raw func → dilate init func brain mask 4. refined_bold_mask : input motion corrected func → dilate anatomical mask 5. get final func mask
# Anatomical_Based: Generate the BOLD mask by basing it off of the anatomical brain mask. Adapted from DCAN Lab's BOLD mask method from the ABCD pipeline.
# Anatomical_Resampled: Resample anatomical brain mask in standard space to get BOLD brain mask in standard space. Adapted from DCAN Lab's BOLD mask method from the ABCD pipeline. ("Create fMRI resolution standard space files for T1w image, wmparc, and brain mask […] don't use FLIRT to do spline interpolation with -applyisoxfm for the 2mm and 1mm cases because it doesn't know the peculiarities of the MNI template FOVs")
# CCS_Anatomical_Refined: Generate the BOLD mask by basing it off of the anatomical brain. Adapted from the BOLD mask method from the CCS pipeline.
# this is a fork point
using: ['FSL']
FSL-BET:
# Set an intensity threshold to improve skull stripping performances of FSL BET on rodent scans.
functional_mean_thr:
run: Off
threshold_value: 98
# Bias correct the functional mean image to improve skull stripping performances of FSL BET on rodent scans
functional_mean_bias_correction: On
# Apply to 4D FMRI data, if bold_bet_functional_mean_boolean : Off.
# Mutually exclusive with functional, reduce_bias, robust, padding, remove_eyes, surfaces
# It must be 'on' if select 'reduce_bias', 'robust', 'padding', 'remove_eyes', or 'bet_surfaces' on
functional_mean_boolean: Off
# Set the threshold value controling the brain vs non-brain voxels.
frac: 0.3
# Mesh created along with skull stripping
mesh_boolean: Off
# Create a surface outline image
outline: Off
# Add padding to the end of the image, improving BET.Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
padding: Off
# Integer value of head radius
radius: 0
# Reduce bias and cleanup neck. Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
reduce_bias: Off
# Eyes and optic nerve cleanup. Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
remove_eyes: Off
# Robust brain center estimation. Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
robust: Off
# Create a skull image
skull: Off
# Gets additional skull and scalp surfaces by running bet2 and betsurf. This is mutually exclusive with reduce_bias, robust, padding, remove_eyes
surfaces: Off
# Apply thresholding to segmented brain image and mask
threshold: Off
# Vertical gradient in fractional intensity threshold (-1,1)
vertical_gradient: 0.0
FSL_AFNI:
brain_mask: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz
brain_probseg: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz
Anatomical_Refined:
# Choose whether or not to dilate the anatomical mask if you choose 'Anatomical_Refined' as the functional masking option. It will dilate one voxel if enabled.
anatomical_mask_dilation: Off
# Apply functional mask in native space
apply_func_mask_in_native_space: On
generate_func_mean:
# Generate mean functional image
run: On
truncation:
# First timepoint to include in analysis.
# Default is 0 (beginning of timeseries).
# First timepoint selection in the scan parameters in the data configuration file, if present, will over-ride this selection.
# Note: the selection here applies to all scans of all participants.
start_tr: 2
despiking:
# Run AFNI 3dDespike
# this is a fork point
# run: [On, Off] - this will run both and fork the pipeline
run: [On]
space: template
nuisance_corrections:
2-nuisance_regression:
# this is a fork point
# run: [On, Off] - this will run both and fork the pipeline
run: [On]
# Select which nuisance signal corrections to apply
Regressors:
- Name: 36P
Bandpass:
bottom_frequency: 0.01
top_frequency: 0.1
CerebrospinalFluid:
erode_mask: Off
extraction_resolution: 2
include_delayed: On
include_delayed_squared: On
include_squared: On
summary: Mean
GlobalSignal:
include_delayed: On
include_delayed_squared: On
include_squared: On
summary: Mean
Motion:
include_delayed: On
include_delayed_squared: On
include_squared: On
WhiteMatter:
erode_mask: Off
extraction_resolution: 2
include_delayed: On
include_delayed_squared: On
include_squared: On
summary: Mean
- Name: aCompCor24P
Bandpass:
bottom_frequency: 0.01
top_frequency: 0.1
Motion:
include_delayed: On
include_delayed_squared: On
include_squared: On
aCompCor:
summary:
method: DetrendPC
components: 5
tissues:
- WhiteMatter
- CerebrospinalFluid
extraction_resolution: 2
# Process and refine masks used to produce regressors and time series for
# regression.
regressor_masks:
erode_csf:
# Erode cerebrospinal fluid mask in millimeters, default for cerebrospinal fluid is 30mm
# Cerebrospinal fluid erosion default is using millimeters.
csf_mask_erosion_mm:
# Target volume ratio, if using erosion.
# Default proportion is None for cerebrospinal fluid mask.
# If using erosion, using both proportion and millimeters is not recommended.
csf_erosion_prop: 0.9
timeseries_extraction:
run: On
connectivity_matrix:
# Create a connectivity matrix from timeseries data
# Options:
# ['AFNI', 'Nilearn', 'ndmg']
using: [Nilearn]
# Enter paths to region-of-interest (ROI) NIFTI files (.nii or .nii.gz) to be used for time-series extraction, and then select which types of analyses to run.
# Denote which analyses to run for each ROI path by listing the names below. For example, if you wish to run Avg and SpatialReg, you would enter: '/path/to/ROI.nii.gz': Avg, SpatialReg
# available analyses:
# /path/to/atlas.nii.gz: Avg, Voxel, SpatialReg
tse_roi_paths:
/ndmg_atlases/label/Human/Glasser_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
/cpac_templates/Schaefer2018_space-FSLMNI152_res-2mm_desc-200Parcels17NetworksOrder.nii.gz: Avg
/cpac_templates/Schaefer2018_space-FSLMNI152_res-2mm_desc-400Parcels17NetworksOrder.nii.gz: Avg
/ndmg_atlases/label/Human/Yeo-17-liberal_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
/ndmg_atlases/label/Human/Yeo-17_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
/ndmg_atlases/label/Human/Yeo-7-liberal_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
/ndmg_atlases/label/Human/Yeo-7_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
/cpac_templates/Sch400TianS1.nii.gz: Avg
/cpac_templates/Sch400TianS2.nii.gz: Avg
/cpac_templates/Sch400TianS3.nii.gz: Avg
/cpac_templates/Sch400TianS4.nii.gz: Avg
amplitude_low_frequency_fluctuation:
# ALFF & f/ALFF
# Calculate Amplitude of Low Frequency Fluctuations (ALFF) and fractional ALFF (f/ALFF) for all voxels.
run: Off
# space: Template or Native
target_space: [Template]
regional_homogeneity:
# ReHo
# Calculate Regional Homogeneity (ReHo) for all voxels.
run: Off
# space: Template or Native
target_space: [Template]
network_centrality:
# Calculate Degree, Eigenvector Centrality, or Functional Connectivity Density.
run: Off
# Full path to a NIFTI file describing the mask. Centrality will be calculated for all voxels within the mask.
template_specification_file: /cpac_templates/Mask_ABIDE_85Percent_GM.nii.gz
degree_centrality:
# Enable/Disable degree centrality by selecting the connectivity weights
# weight_options: ['Binarized', 'Weighted']
# disable this type of centrality with:
# weight_options: []
weight_options: [Binarized]
eigenvector_centrality:
# Enable/Disable eigenvector centrality by selecting the connectivity weights
# weight_options: ['Binarized', 'Weighted']
# disable this type of centrality with:
# weight_options: []
weight_options: []
local_functional_connectivity_density:
# Enable/Disable lFCD by selecting the connectivity weights
# weight_options: ['Binarized', 'Weighted']
# disable this type of centrality with:
# weight_options: []
weight_options: [Binarized, Weighted]
# OUTPUTS AND DERIVATIVES
# -----------------------
post_processing:
spatial_smoothing:
# Smooth the derivative outputs.
# Set as ['nonsmoothed'] to disable smoothing. Set as ['smoothed', 'nonsmoothed'] to get both.
#
# Options:
# ['smoothed', 'nonsmoothed']
output: [smoothed, nonsmoothed]
# Tool to use for smoothing.
# 'FSL' for FSL MultiImageMaths for FWHM provided
# 'AFNI' for AFNI 3dBlurToFWHM for FWHM provided
smoothing_method: [AFNI]
# Full Width at Half Maximum of the Gaussian kernel used during spatial smoothing.
# this is a fork point
# i.e. multiple kernels - fwhm: [4,6,8]
fwhm: [4,6]
z-scoring:
# z-score standardize the derivatives. This may be needed for group-level analysis.
# Set as ['raw'] to disable z-scoring. Set as ['z-scored', 'raw'] to get both.
#
# Options:
# ['z-scored', 'raw']
output: [z-scored, raw]
Run command
#!/usr/bin/bash
#SBATCH --nodes=1
#SBATCH --job-name=prek_rbc
#SBATCH --partition=RM-shared
#SBATCH --ntasks=10
#SBATCH --time=24:00:00
#SBATCH --error=/ocean/projects/med220004p/jclucas/UserSupport/Shinwon/longitudinal/batch_log/1_%x_%A_%a.err
#SBATCH --output=/ocean/projects/med220004p/jclucas/UserSupport/Shinwon/longitudinal/batch_log/1_%x_%A_%a.out
#SBATCH [email protected]
#SBATCH --mail-type=BEGIN
#SBATCH --mail-type=END
#SBATCH --mail-type=FAIL
export DATA="/ocean/projects/med220004p/spark9/02_data/prek_updated_10m/"
export IMAGE="/ocean/projects/med220004p/spark9/01_project/01_prek/02_preproc_all/03_longitudinal_10m/c-pac:release-v1.8.7.post1.dev2.sif"
export ROOT_DIR="/ocean/projects/med220004p/jclucas/UserSupport/Shinwon/longitudinal"
export SUBJECT="70009"
export NCPUS=1 # Disable nipype graph parallelism
export NTHREADS=10
export MEMGB=20 # 2x --ntasks; PSC provisions 2GB per cpu
CMD="apptainer run -B /ocean/projects/med220004p/jclucas/C-PAC/CPAC:/code/CPAC:ro -B $DATA:$DATA:ro -B $ROOT_DIR:$ROOT_DIR $IMAGE $DATA $ROOT_DIR/outputs participant --participant_label $SUBJECT --n_cpus=$NCPUS --num_ants_threads $NTHREADS --skip_bids_validator --save_working_dir --fail_fast True --mem-gb $MEMGB --pipeline_file=$ROOT_DIR/pipeline_config_rbc-options_addRegressors_noBBR_bugfix_funcmask_FSL_long.yml"
echo $CMD
eval $CMD
exitcode=$?
exit exitcode
Expected behavior
Longitudinal pipeline runs or a helpful error message is provided.
Acceptance criteria
- longitudinal pipeline runs to completion
C-PAC version
v1.8.7.post1.dev2
Container platform
Apptainer
Docker and/or Singularity version(s)
Apptainer v1.3.4-1.el8
Additional context
Reported by @Shinwon Park on Slack
Metadata
Metadata
Assignees
Type
Projects
Status
🏗 In progress