Skip to content

ENH: Confound model enhancement #1586

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 57 commits into from
May 4, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
513be36
(#1458) sketch CompCor enhancement
rciric Jan 26, 2019
ce35dd6
(#1458) new reportlets for CompCor and confounds
rciric Jan 26, 2019
6abb4da
Merge branch 'master' of https://github.com/poldracklab/fmriprep
rciric Jan 26, 2019
ce4d130
jsonise and return CompCor metadata
rciric Jan 27, 2019
64a65ee
jsonise and return CompCor metadata
rciric Jan 27, 2019
0008fb1
add references for expanded model and censoring thresholds
rciric Jan 27, 2019
4d5d614
acompcor metadata to circle
rciric Jan 28, 2019
71ff4dc
add contributor
rciric Jan 28, 2019
6ffe441
tcompcor metadata to circle
rciric Jan 28, 2019
ebfa9f9
fix figure legends
rciric Jan 28, 2019
95dcb2a
more descriptive citation
rciric Jan 31, 2019
1e50ae3
run aCompCor for WM, CSF, and combined masks
rciric Jan 31, 2019
22c9e4b
specification of spike thresholds and decomposition output
rciric Feb 8, 2019
3134728
propagate spike and CompCor specs
rciric Feb 8, 2019
b99afb0
add CLI options for spikes and CompCor
rciric Feb 8, 2019
5618629
[skip ci] expand description, minor fixes
rciric Feb 8, 2019
3a0d149
[skip ci] conform to updated SpikeRegressors interface
rciric Feb 17, 2019
50b8ce2
Merge remote-tracking branch 'upstream/master' into rciric-patch-1
oesteban Feb 21, 2019
0b32336
pin adequate versions of nipype and niworkflows
oesteban Feb 21, 2019
38d0550
Merge pull request #1 from oesteban/rciric-patch-1
rciric Feb 21, 2019
d9c33e9
[skip ci] update documentation to reflect new confounds options
rciric Feb 22, 2019
69a7908
[skip ci] update documentation to reflect new confounds options
rciric Feb 22, 2019
ae1d91d
[skip ci] update documentation to reflect new confounds options
rciric Feb 22, 2019
b0a7490
use NaN failure mode for empty masks [skip ci]
rciric Mar 27, 2019
f5b9e35
sketch confounds metadata sink
rciric Apr 10, 2019
caa8a0e
reconcile updates
rciric Apr 13, 2019
04ec5c9
configure circle wf
rciric Apr 15, 2019
e1fa3e4
configure circle wf
rciric Apr 15, 2019
9419f00
update circle hash to latest niworkflows branch
rciric Apr 17, 2019
4532698
Merge branch 'master' into master
effigies Apr 17, 2019
834ad54
reconcile updates
rciric Apr 17, 2019
08f23f3
Merge branch 'master' of https://github.com/rciric/fmriprep
rciric Apr 17, 2019
9f1194a
fix(deps): pin correct nipype commit, fix duplicated niworkflows link
oesteban Apr 17, 2019
31188bd
Merge branch 'ds005/fix-master' into rmaster
oesteban Apr 17, 2019
7ad3602
Merge remote-tracking branch 'rciric/verify_unpushed' into pr/1487
oesteban Apr 17, 2019
1038b21
fix(autodocs): remove extra parens in ``.. workflow::`` block of ``in…
oesteban Apr 17, 2019
44708bc
fix(tests): add output missing in partial derivatives index [skip doc…
oesteban Apr 17, 2019
4a7b5ff
update pin for dependencies
rciric Apr 19, 2019
6bebccc
(maint) reconcile with upstream
rciric Apr 19, 2019
d16e751
(maint) reconcile with upstream
rciric Apr 19, 2019
59ee4b6
Merge remote-tracking branch 'upstream/master' into pr/1487
oesteban Apr 24, 2019
8fe2f75
Merge branch 'pr/1487' into repin
oesteban Apr 24, 2019
7e24bb6
Merge pull request #3 from rciric/repin
oesteban Apr 24, 2019
28c0823
Merge remote-tracking branch 'upstream/master' into pr/1487
oesteban Apr 29, 2019
6af536b
pin(niworkflows,nipype): update pinnings before merge
oesteban Apr 29, 2019
c76a5a1
Merge branch 'pr/1487' of github.com:oesteban/fmriprep into pr/1487
oesteban Apr 29, 2019
3db3156
Merge remote-tracking branch 'upstream/master' into pr/1487
oesteban May 2, 2019
ea4726a
Merge branch 'enh/new-reports' into pr/1487
oesteban May 2, 2019
50c8287
fix(workflow): set correct ``desc`` input to ``DerivativesDataSink``
oesteban May 2, 2019
b74dddb
Merge remote-tracking branch 'upstream/master' into pr/1487
oesteban May 2, 2019
463e237
pin(niworkflows): update to latest poldracklab/niworkflows#357
oesteban May 3, 2019
16bebee
Merge remote-tracking branch 'upstream/master' into pr/1487
oesteban May 3, 2019
38378e8
Merge remote-tracking branch 'upstream/master' into pr/1487
oesteban May 3, 2019
4c22d7f
fix(workflow): address @effigies' comment
oesteban May 4, 2019
325c6a9
fix(workflow): missing connections forgotten when resolving conflicts
oesteban May 4, 2019
6a62c86
fix(docs): add missing comma after function argument
oesteban May 4, 2019
b6e090e
Merge remote-tracking branch 'upstream/master' into pr/1487
oesteban May 4, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .circleci/ds005_outputs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ fmriprep/sub-01/func
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_AROMAnoiseICs.csv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold.dtseries.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_bold.dtseries.nii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_desc-confounds_regressors.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_desc-confounds_regressors.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_desc-MELODIC_mixing.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_space-fsaverage5_hemi-L.func.gii
Expand Down Expand Up @@ -85,6 +86,7 @@ fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-01_space-T1w_desc-preproc_
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_AROMAnoiseICs.csv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold.dtseries.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold.dtseries.nii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_desc-confounds_regressors.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_desc-confounds_regressors.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_desc-MELODIC_mixing.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_space-fsaverage5_hemi-L.func.gii
Expand Down
1 change: 1 addition & 0 deletions .circleci/ds005_partial_outputs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ fmriprep/sub-01/func
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_AROMAnoiseICs.csv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold.dtseries.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_bold.dtseries.nii
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_desc-confounds_regressors.json
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_desc-confounds_regressors.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_desc-MELODIC_mixing.tsv
fmriprep/sub-01/func/sub-01_task-mixedgamblestask_run-02_space-fsaverage5_hemi-L.func.gii
Expand Down
2 changes: 2 additions & 0 deletions .circleci/ds054_outputs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ fmriprep/sub-100185/anat/sub-100185_space-MNI152NLin2009cAsym_label-CSF_probseg.
fmriprep/sub-100185/anat/sub-100185_space-MNI152NLin2009cAsym_label-GM_probseg.nii.gz
fmriprep/sub-100185/anat/sub-100185_space-MNI152NLin2009cAsym_label-WM_probseg.nii.gz
fmriprep/sub-100185/func
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_desc-confounds_regressors.json
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_desc-confounds_regressors.tsv
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_space-MNI152NLin2009cAsym_boldref.nii.gz
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.json
Expand All @@ -38,6 +39,7 @@ fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_space-T1w_desc-brain
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_space-T1w_desc-brain_mask.nii.gz
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_space-T1w_desc-preproc_bold.json
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-01_space-T1w_desc-preproc_bold.nii.gz
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-02_desc-confounds_regressors.json
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-02_desc-confounds_regressors.tsv
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-02_space-MNI152NLin2009cAsym_boldref.nii.gz
fmriprep/sub-100185/func/sub-100185_task-machinegame_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.json
Expand Down
1 change: 1 addition & 0 deletions .circleci/ds210_outputs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ fmriprep/sub-02/anat/sub-02_space-MNI152NLin2009cAsym_label-CSF_probseg.nii.gz
fmriprep/sub-02/anat/sub-02_space-MNI152NLin2009cAsym_label-GM_probseg.nii.gz
fmriprep/sub-02/anat/sub-02_space-MNI152NLin2009cAsym_label-WM_probseg.nii.gz
fmriprep/sub-02/func
fmriprep/sub-02/func/sub-02_task-cuedSGT_run-01_desc-confounds_regressors.json
fmriprep/sub-02/func/sub-02_task-cuedSGT_run-01_desc-confounds_regressors.tsv
fmriprep/sub-02/func/sub-02_task-cuedSGT_run-01_space-MNI152NLin2009cAsym_boldref.nii.gz
fmriprep/sub-02/func/sub-02_task-cuedSGT_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.json
Expand Down
5 changes: 5 additions & 0 deletions .zenodo.json
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,11 @@
"affiliation": "Neuroscience Program, University of Iowa",
"orcid": "0000-0002-3339-4857"
},
{
"name": "Ciric, Rastko",
"affiliation": "Stanford University",
"orcid": "0000-0001-6347-7939"
},
{
"name": "Poldrack, Russell A.",
"affiliation": "Department of Psychology, Stanford University",
Expand Down
2 changes: 1 addition & 1 deletion docs/outputs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ motion-correction parameters estimated by fMRIPrep;
if present, ``non_steady_state_outlier_XX`` columns indicate non-steady state volumes with a single
``1`` value and ``0`` elsewhere (*i.e.*, there is one ``non_steady_state_outlier_XX`` column per
outlier/volume);
six noise components are calculated using :abbr:`CompCor (Component Based Noise Correction)`,
additional noise components are calculated using :abbr:`CompCor (Component Based Noise Correction)`,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rciric I feel we will need to expand this and elaborate a bit more about what components were dropped, how they are denoted and why they were dropped. I'd be fine to do that on another PR (with a branch called docs/<whatever> we would be saving a lot of CircleCI cycles).

according to both the anatomical (``a_comp_cor_XX``) and temporal (``t_comp_cor_XX``) variants;
and the motion-related components identified by
:abbr:`ICA (independent components analysis)`-:abbr:`AROMA (Automatic Removal Of Motion Artifacts)`
Expand Down
12 changes: 11 additions & 1 deletion docs/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ is presented below:
('MNI152Lin', {}), ('fsaverage', {'density': '10k'}),
('T1w', {}), ('fsnative', {})]),
reportlets_dir='.',
regressors_all_comps=False,
regressors_dvars_th=1.5,
regressors_fd_th=0.5,
skull_strip_fixed_seed=False,
skull_strip_template='OASIS30ANTs',
subject_id='test',
Expand Down Expand Up @@ -299,6 +302,9 @@ BOLD preprocessing
reportlets_dir='.',
t2s_coreg=False,
use_aroma=False,
regressors_all_comps=False,
regressors_fd_th=0.5,
regressors_dvars_th=1.5,
use_bbr=True,
use_syn=True,
layout=BIDSLayout('.'),
Expand Down Expand Up @@ -543,7 +549,11 @@ Confounds estimation
name="discover_wf",
mem_gb=1,
metadata={"RepetitionTime": 2.0,
"SliceTiming": [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]})
"SliceTiming": [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]},
regressors_all_comps=False,
regressors_dvars_th=1.5,
regressors_fd_th=0.5,
)

Given a motion-corrected fMRI, a brain mask, ``mcflirt`` movement parameters and a
segmentation, the `discover_wf` sub-workflow calculates potential
Expand Down
5 changes: 4 additions & 1 deletion fmriprep/__about__.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@
'indexed_gzip>=0.8.8',
'nibabel>=2.2.1',
'nilearn!=0.5.0,!=0.5.1',
'nipype>=1.1.6',
'nipype @ git+https://github.com/nipy/nipype.git@'
'd353f0d879826031334b09d33e9443b8c9b3e7fe',
'nitime',
'niworkflows<0.10.0a0,>=0.9.1.post1',
'numpy',
Expand All @@ -103,6 +104,8 @@


LINKS_REQUIRES = [
'git+https://github.com/nipy/nipype.git@'
'd353f0d879826031334b09d33e9443b8c9b3e7fe#egg=nipype',
]

TESTS_REQUIRES = [
Expand Down
19 changes: 19 additions & 0 deletions fmriprep/cli/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,22 @@ def get_parser():
help='Exact or maximum number of MELODIC components to estimate '
'(positive = exact, negative = maximum)')

# Confounds options
g_confounds = parser.add_argument_group('Specific options for estimating confounds')
g_confounds.add_argument(
'--return-all-components', required=False, action='store_true', default=False,
help='Include all components estimated in CompCor decomposition in the confounds '
'file instead of only the components sufficient to explain 50 percent of '
'BOLD variance in each CompCor mask')
g_confounds.add_argument(
'--fd-spike-threshold', required=False, action='store', default=0.5, type=float,
help='Threshold for flagging a frame as an outlier on the basis of framewise '
'displacement')
g_confounds.add_argument(
'--dvars-spike-threshold', required=False, action='store', default=1.5, type=float,
help='Threshold for flagging a frame as an outlier on the basis of standardised '
'DVARS')

# ANTs options
g_ants = parser.add_argument_group('Specific options for ANTs registrations')
g_ants.add_argument('--skull-strip-template', action='store', default='OASIS30ANTs',
Expand Down Expand Up @@ -574,6 +590,9 @@ def build_workflow(opts, retval):
output_dir=str(output_dir),
output_spaces=output_spaces,
run_uuid=run_uuid,
regressors_all_comps=opts.return_all_components,
regressors_fd_th=opts.fd_spike_threshold,
regressors_dvars_th=opts.dvars_spike_threshold,
skull_strip_fixed_seed=opts.skull_strip_fixed_seed,
skull_strip_template=opts.skull_strip_template,
subject_list=subject_list,
Expand Down
14 changes: 14 additions & 0 deletions fmriprep/data/boilerplate.bib
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,20 @@ @article{power_fd_dvars
year = 2014
}

@article{confounds_satterthwaite_2013,
author = {Satterthwaite, Theodore D. and Elliott, Mark A. and Gerraty, Raphael T. and Ruparel, Kosha and Loughead, James and Calkins, Monica E. and Eickhoff, Simon B. and Hakonarson, Hakon and Gur, Ruben C. and Gur, Raquel E. and Wolf, Daniel H.},
doi = {10.1016/j.neuroimage.2012.08.052},
issn = {10538119},
journal = {NeuroImage},
number = 1,
pages = {240--256},
title = {{An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data}},
url = {http://linkinghub.elsevier.com/retrieve/pii/S1053811912008609},
volume = 64,
year = 2013
}


@article{nilearn,
author = {Abraham, Alexandre and Pedregosa, Fabian and Eickenberg, Michael and Gervais, Philippe and Mueller, Andreas and Kossaifi, Jean and Gramfort, Alexandre and Thirion, Bertrand and Varoquaux, Gael},
doi = {10.3389/fninf.2014.00014},
Expand Down
Loading