Fieldmaps are not properly collated for session-wise processing

[JohannesWiesner]

What happened?

I use fmriprep 25.2.3 together with the --subject-anatomical-reference sessionwise argument to preprocess my subject data session-wise. The reason for this is that the scanner settings have changed throughout the study. fmriprep finishes succesfully but for the sessions (where the scanner parameter have changed) it does not use fieldmap correction.

What command did you use?

sbatch \
 --job-name fmriprep-25.2.3_sub-6IECX \
 --output /zi/home/johannes.wiesner/work/slurm/project_bernn/log_files/fmriprep/fmriprep-25.2.3_sub-6IECX_job-%j.out \
 --cpus-per-task 16 \
 --mem 64G \
 --mail-user johannes.wiesner@zi-mannheim.de \
 --mail-type END \
 --partition snip,cpu \
 --chdir /zi/home/johannes.wiesner \
 --wrap "bash -c 'export KRB5CCNAME=FILE:/tmp/krb5cc_\$(id -u)_\${SLURM_JOB_ID} && \
  kinit johannes.wiesner@ZI.LOCAL -k -t /zi/home/johannes.wiesner/johannes.wiesner.keytab && \
  krenew -b -K 60 && \
  export TMPDIR=/data/\${SLURM_JOB_ID} && \
  mkdir /zi/home/johannes.wiesner/work/slurm/project_bernn/tmp/\${SLURM_JOB_ID}_fmriprep && \
  module load fmriprep/25.2.3 && \
  fmriprep /zi/flstorage/group_csp/in_house_datasets/bernn/mri/rawdata/ /zi/home/johannes.wiesner/work/slurm/project_bernn/tmp/\${SLURM_JOB_ID}_fmriprep participant \
 --fs-license-file /zi/home/johannes.wiesner/work/it/freesurfer_license.txt \
 --n_cpus 16 \
 --skip-bids-validation  \
 --work-dir /zi/home/johannes.wiesner/work/slurm/project_bernn/work_dirs/fmriprep/ \
 --fs-no-reconall  \
 --subject-anatomical-reference sessionwise \
 --participant-label 6IECX && \
  while ! rsync -av --ignore-existing /zi/home/johannes.wiesner/work/slurm/project_bernn/tmp/\${SLURM_JOB_ID}_fmriprep/ /zi/flstorage/group_csp/in_house_datasets/bernn/mri/derivatives/fmriprep_25.2.3;do sleep 5;done && \
  rm -rf /zi/home/johannes.wiesner/work/slurm/project_bernn/tmp/\${SLURM_JOB_ID}_fmriprep'"

What version of fMRIPrep are you running?

25.2.3

How are you running fMRIPrep?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

Additional information / screenshots

These are the fieldmap sidecars for the first session (where fieldmap distortion correction is applied)

sub-YL4AS_ses-01_dir-AP_epi.json
sub-YL4AS_ses-01_dir-PA_epi.json

And these are the ones for session 5 where the scanner settings have changed and where fieldmap correction is not applied:

sub-YL4AS_ses-05_dir-AP_epi.json
sub-YL4AS_ses-05_dir-PA_epi.json

I used https://www.diffchecker.com/text-compare/ to compare the respective AP/PA sidecars for ses-01 vs. ses-05 but could not find out if there's something wrong in the sidecars of ses-05 that would hinder fmriprep from running fieldmap correction for this session

Here's the full log:

fmriprep-25.2.3_sub-YL4AS_job-675470.txt

[effigies]

Can you try running sdcflows on your dataset?

uvx -p3.13 sdcflows $DATA /tmp/fake-out participant --dry-run -w /tmp/fake-wd --participant-label 6IECX

[JohannesWiesner]

This is a separate container right? Just the latest version?

[effigies]

You can use the same container, actually. Then just drop the uvx -p bit.

[JohannesWiesner]

Results show that it does find all the fieldmaps for all sessions:

(wiesner_johannes) johannes.wiesner@zislrds0024:~/work/slurm/testing$ singularity shell \
-B /zi/flstorage/group_csp/in_house_datasets/bernn/mri/rawdata/:/data:ro \
/zi/apps/container/fmriprep_25.2.3.sif
Singularity> sdcflows --help
usage: sdcflows [-h] [--version] [-v] [--participant-label PARTICIPANT_LABEL [PARTICIPANT_LABEL ...]] [--session-label [SESSION_LABEL ...]] [--bids-filter-file PATH] [--bids-database-dir PATH] [--bids-database-wipe] [--nprocs NPROCS] [--omp-nthreads OMP_NTHREADS] [--mem-gb MEMORY_GB] [--debug] [--pdb] [--sloppy] [-w WORK_DIR] [-n] [--no-fmapless]
                [--use-plugin USE_PLUGIN] [--notrack]
                bids_dir output_dir {participant,group} [{participant,group} ...]

SDCFlows 2.15.0 Estimate fieldmaps available in a BIDS-compliant MRI dataset.

positional arguments:
  bids_dir              The root folder of a BIDS valid dataset (sub-XXXXX folders should be found at the top level in this folder).
  output_dir            The directory where the output files should be stored. If you are running group level analysis this folder should be prepopulated with the results of the participant level analysis.
  {participant,group}   Level of the analysis that will be performed. Multiple participant level analyses can be run independently (in parallel) using the same output_dir.

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v, --verbose         Increases log verbosity for each occurrence, debug level is -vvv. (default: 0)

Options related to BIDS:
  --participant-label PARTICIPANT_LABEL [PARTICIPANT_LABEL ...]
                        A space delimited list of participant identifiers or a single identifier (the sub- prefix can be removed). (default: None)
  --session-label [SESSION_LABEL ...]
                        Filter input dataset by session label. (default: None)
  --bids-filter-file PATH
                        a JSON file describing custom BIDS input filter using pybids {<suffix>:{<entity>:<filter>,...},...} (https://github.com/bids-standard/pybids/blob/master/bids/layout/config/bids.json) (default: None)
  --bids-database-dir PATH
                        Path to an existing PyBIDS database folder, for faster indexing (especially useful for large datasets). (default: None)
  --bids-database-wipe  Wipe out previously existing BIDS indexing caches, forcing re-indexing. (default: False)

Options to handle performance:
  --nprocs NPROCS       Maximum number of simultaneously running parallel processes executed by *SDCFlows* (e.g., several instances of ANTs' registration). However, when ``--nprocs`` is greater or equal to the ``--omp-nthreads`` option, it also sets the maximum number of threads that simultaneously running processes may aggregate (meaning, with
                        ``--nprocs 16 --omp-nthreads 8`` a maximum of two 8-CPU-threaded processes will be running at a given time). Under this mode of operation, ``--nprocs`` sets the maximum number of processors that can be assigned work within an *SDCFlows* job, which includes all the processors used by currently running single- and multi-threaded
                        processes. If ``None``, the number of CPUs available will be automatically assigned (which may not be what you want in, e.g., shared systems like a HPC cluster. (default: None)
  --omp-nthreads OMP_NTHREADS
                        Maximum number of threads that multi-threaded processes executed by *SDCFlows* (e.g., ANTs' registration) can use. If ``None``, the number of CPUs available will be automatically assigned (which may not be what you want in, e.g., shared systems like a HPC cluster. (default: None)
  --mem-gb MEMORY_GB    Upper bound memory limit for SDCFlows processes. (default: None)
  --debug               Enable changes to processing to aid in debugging (default: False)
  --pdb                 Open Python debugger (pdb) on exceptions. (default: False)
  --sloppy              Use sloppy mode for minimal footprint. (default: False)

Instrumental options:
  -w WORK_DIR, --work-dir WORK_DIR
                        Path where intermediate results should be stored. (default: /zi/home/johannes.wiesner/work/slurm/testing/work)
  -n, --dry-run         only find estimable fieldmaps (that is, estimation is not triggered) (default: False)
  --no-fmapless         Allow fieldmap-less estimation (default: True)
  --use-plugin USE_PLUGIN
                        Nipype plugin configuration file. (default: None)
  --notrack             Opt-out of sending tracking information of this run to the NiPreps developers. This information helps to improve SDCFlows and provides an indicator of real world usage for obtaining funding. (default: False)
Singularity> sdcflows /data /tmp/fake-out participant --dry-run -w /tmp/fake-wd --participant-label 6IECX
Estimation for </data> complete. Found:
        sub-6IECX
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00000')
                        j-      ses-01/fmap/sub-6IECX_ses-01_acq-dwi_dir-AP_epi.nii.gz
                        j       ses-01/fmap/sub-6IECX_ses-01_acq-dwi_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00001')
                        j-      ses-01/fmap/sub-6IECX_ses-01_dir-AP_epi.nii.gz
                        j       ses-01/fmap/sub-6IECX_ses-01_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00002')
                        j-      ses-02/fmap/sub-6IECX_ses-02_acq-dwi_dir-AP_epi.nii.gz
                        j       ses-02/fmap/sub-6IECX_ses-02_acq-dwi_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00003')
                        j-      ses-02/fmap/sub-6IECX_ses-02_dir-AP_epi.nii.gz
                        j       ses-02/fmap/sub-6IECX_ses-02_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00004')
                        j-      ses-03/fmap/sub-6IECX_ses-03_acq-dwi_dir-AP_epi.nii.gz
                        j       ses-03/fmap/sub-6IECX_ses-03_acq-dwi_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00005')
                        j-      ses-03/fmap/sub-6IECX_ses-03_dir-AP_epi.nii.gz
                        j       ses-03/fmap/sub-6IECX_ses-03_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00006')
                        j-      ses-04/fmap/sub-6IECX_ses-04_acq-dwi_dir-AP_epi.nii.gz
                        j       ses-04/fmap/sub-6IECX_ses-04_acq-dwi_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00007')
                        j-      ses-04/fmap/sub-6IECX_ses-04_dir-AP_epi.nii.gz
                        j       ses-04/fmap/sub-6IECX_ses-04_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00008')
                        j-      ses-05/fmap/sub-6IECX_ses-05_acq-dwi_dir-AP_epi.nii.gz
                        j       ses-05/fmap/sub-6IECX_ses-05_acq-dwi_dir-PA_epi.nii.gz
                FieldmapEstimation(sources=<2 files>, method=<EstimatorType.PEPOLAR: 2>, bids_id='auto_00009')
                        j-      ses-05/fmap/sub-6IECX_ses-05_dir-AP_epi.nii.gz
                        j       ses-05/fmap/sub-6IECX_ses-05_dir-PA_epi.nii.gz
Singularity> 

[JohannesWiesner]

Update: I am not sure anymore if it has to do with scanner setting change. So far, for all my subjects only the first session is distortion corrected (regardless of whether scanner parameters have changed or not)

[effigies]

Yeah, we probably didn't handle the fieldmap wrangler properly for sessionwise processing. I suspect if you pass --session-label $SES, each one will work correctly.

[JohannesWiesner]

Yeah, we probably didn't handle the fieldmap wrangler properly for sessionwise processing. I suspect if you pass --session-label $SES, each one will work correctly.

Can confirm. Processing each subject-session-combination separately exits without errors.