Skip to content

Commit 278fb84

Browse files
oestebanoesteban
authored
Merge pull request #1802 from oesteban/docs/templateflow-1801
DOC: Improve documentation about TemplateFlow and Containers
2 parents 1c4f2ab + 2800d49 commit 278fb84

File tree

14 files changed

+806
-292
lines changed

14 files changed

+806
-292
lines changed

‎README.rst

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
fmriprep: A Robust Preprocessing Pipeline for fMRI Data
2-
=======================================================
1+
*fMRIPrep*: A Robust Preprocessing Pipeline for fMRI Data
2+
=========================================================
33

44
This pipeline is developed by the `Poldrack lab at Stanford University
55
<https://poldracklab.stanford.edu/>`_ for use at the `Center for Reproducible
@@ -43,7 +43,7 @@ About
4343
.. image:: https://github.com/oesteban/fmriprep/raw/38a63e9504ab67812b63813c5fe9af882109408e/docs/_static/fmriprep-workflow-all.png
4444

4545

46-
``fmriprep`` is a functional magnetic resonance imaging (fMRI) data
46+
*fMRIPrep* is a functional magnetic resonance imaging (fMRI) data
4747
preprocessing pipeline that is designed to provide an easily accessible,
4848
state-of-the-art interface that is robust to variations in scan acquisition
4949
protocols and that requires minimal user input, while providing easily
@@ -56,12 +56,12 @@ volume-based statistics, etc.
5656

5757
.. note::
5858

59-
fmriprep performs minimal preprocessing.
59+
*fMRIPrep* performs minimal preprocessing.
6060
Here we define 'minimal preprocessing' as motion correction, field
6161
unwarping, normalization, bias field correction, and brain extraction.
6262
See the workflows_ for more details.
6363

64-
The ``fmriprep`` pipeline uses a combination of tools from well-known software
64+
The *fMRIPrep* pipeline uses a combination of tools from well-known software
6565
packages, including FSL_, ANTs_, FreeSurfer_ and AFNI_.
6666
This pipeline was designed to provide the best software implementation for each
6767
state of preprocessing, and will be updated as newer and better neuroimaging
@@ -86,7 +86,7 @@ https://fmriprep.readthedocs.io/
8686
Principles
8787
----------
8888

89-
``fmriprep`` is built around three principles:
89+
*fMRIPrep* is built around three principles:
9090

9191
1. **Robustness** - The pipeline adapts the preprocessing steps depending on
9292
the input dataset and should provide results as good as possible
@@ -97,22 +97,22 @@ Principles
9797
automatic fashion.
9898
3. **"Glass box"** philosophy - Automation should not mean that one should not
9999
visually inspect the results or understand the methods.
100-
Thus, ``fmriprep`` provides visual reports for each subject, detailing the
100+
Thus, *fMRIPrep* provides visual reports for each subject, detailing the
101101
accuracy of the most important processing steps.
102102
This, combined with the documentation, can help researchers to understand
103103
the process and decide which subjects should be kept for the group level
104104
analysis.
105105

106106

107-
Limitations and reasons not to use ``fmriprep``
108-
-----------------------------------------------
107+
Limitations and reasons not to use *fMRIPrep*
108+
---------------------------------------------
109109

110110
1. Very narrow :abbr:`FoV (field-of-view)` images oftentimes do not contain
111111
enough information for standard image registration methods to work correctly.
112112
Also, problems may arise when extracting the brain from these data.
113113
Supporting these particular images is already a future line of the development
114114
road-map.
115-
2. ``fmriprep`` may also underperform for particular populations (e.g., infants) and
115+
2. *fMRIPrep* may also underperform for particular populations (e.g., infants) and
116116
non-human brains, although appropriate templates can be provided to overcome the
117117
issue.
118118
3. The "EPInorm" approach is currently not supported, although we plan to implement
@@ -131,6 +131,6 @@ Acknowledgements
131131
----------------
132132

133133
Please acknowledge this work by mentioning explicitly the name of this software
134-
(fmriprep) and the version, along with a link to the `GitHub repository
134+
(*fMRIPrep*) and the version, along with a link to the `GitHub repository
135135
<https://github.com/poldracklab/fmriprep>`_ or the Zenodo reference.
136136
For more details, please see :ref:`citation`.

‎docs/_static/sbatch.slurm

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
#!/bin/bash
2+
#
3+
#SBATCH -J fmriprep
4+
#SBATCH --time=48:00:00
5+
#SBATCH -n 1
6+
#SBATCH --cpus-per-task=16
7+
#SBATCH --mem-per-cpu=4G
8+
#SBATCH -p normal,mygroup # Queue names you can submit to
9+
# Outputs ----------------------------------
10+
#SBATCH -o log/%x-%A-%a.out
11+
#SBATCH -e log/%x-%A-%a.err
12+
#SBATCH --mail-user=%[email protected]
13+
#SBATCH --mail-type=ALL
14+
# ------------------------------------------
15+
16+
BIDS_DIR="$STUDY/data"
17+
DERIVS_DIR="derivatives/fmriprep-1.5.0"
18+
19+
# Prepare some writeable bind-mount points.
20+
TEMPLATEFLOW_HOST_HOME=$HOME/.cache/templateflow
21+
FMRIPREP_HOST_CACHE=$HOME/.cache/fmriprep
22+
mkdir -p ${TEMPLATEFLOW_HOST_HOME}
23+
mkdir -p ${FMRIPREP_HOST_CACHE}
24+
25+
# Prepare derivatives folder
26+
mkdir -p ${BIDS_DIR}/${DERIVS_DIR}
27+
28+
# This trick will help you reuse freesurfer results across pipelines and fMRIPrep versions
29+
mkdir -p ${BIDS_DIR}/derivatives/freesurfer-6.0.1
30+
if [ ! -d ${BIDS_DIR}/${DERIVS_DIR}/freesurfer ]; then
31+
ln -s ${BIDS_DIR}/derivatives/freesurfer-6.0.1 ${BIDS_DIR}/${DERIVS_DIR}/freesurfer
32+
fi
33+
34+
# Make sure FS_LICENSE is defined in the container.
35+
export SINGULARITYENV_FS_LICENSE=$HOME/.freesurfer.txt
36+
37+
# Designate a templateflow bind-mount point
38+
export SINGULARITYENV_TEMPLATEFLOW_HOME="/templateflow"
39+
SINGULARITY_CMD="singularity run --cleanenv -B $BIDS_DIR:/data -B ${TEMPLATEFLOW_HOST_HOME}:${SINGULARITYENV_TEMPLATEFLOW_HOME} -B $L_SCRATCH:/work $STUDY/images/poldracklab_fmriprep_1.5.0.simg"
40+
41+
# Parse the participants.tsv file and extract one subject ID from the line corresponding to this SLURM task.
42+
subject=$( sed -n -E "$((${SLURM_ARRAY_TASK_ID} + 1))s/sub-(\S*)\>.*/\1/gp" ${BIDS_DIR}/participants.tsv )
43+
44+
# Remove IsRunning files from FreeSurfer
45+
find ${BIDS_DIR}/derivatives/freesurfer-6.0.1/sub-$subject/ -name "*IsRunning*" -type f -delete
46+
47+
# Compose the command line
48+
cmd="${SINGULARITY_CMD} /data /data/${DERIVS_DIR} participant --participant-label $subject -w /work/ -vv --omp-nthreads 8 --nthreads 12 --mem_mb 30000 --output-spaces MNI152NLin2009cAsym:res-2 anat fsnative fsaverage5 --use-aroma"
49+
50+
# Setup done, run the command
51+
echo Running task ${SLURM_ARRAY_TASK_ID}
52+
echo Commandline: $cmd
53+
eval $cmd
54+
exitcode=$?
55+
56+
# Output results to a table
57+
echo "sub-$subject ${SLURM_ARRAY_TASK_ID} $exitcode" \
58+
>> ${SLURM_JOB_NAME}.${SLURM_ARRAY_JOB_ID}.tsv
59+
echo Finished tasks ${SLURM_ARRAY_TASK_ID} with exit code $exitcode
60+
exit $exitcode

‎docs/citing.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
.. _citation:
44

5-
===============
6-
Citing FMRIPREP
7-
===============
5+
=================
6+
Citing *fMRIPrep*
7+
=================
88

9-
Select which options you have run FMRIPREP with to generate custom language
9+
Select which options you have run *fMRIPrep* with to generate custom language
1010
we recommend to include in your paper.
1111

1212

‎docs/conf.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@
7676

7777
# General information about the project.
7878
project = 'fmriprep'
79-
author = 'The FMRIPREP developers'
79+
author = 'The fMRIPrep developers'
8080
copyright = '2016-%s, %s' % (datetime.now().year, author)
8181

8282
# The version info for the project you're documenting, acts as replacement for

‎docs/contributors.rst

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,8 @@ Patching working repositories
2424
In order to test new code without rebuilding the Docker image, it is
2525
possible to mount working repositories as source directories within the
2626
container.
27-
The `fmriprep-docker`_ script simplifies this for the most common repositories::
27+
The `Docker wrapper`_ script simplifies this
28+
for the most common repositories::
2829

2930
-f PATH, --patch-fmriprep PATH
3031
working fmriprep repository (default: None)
@@ -131,8 +132,8 @@ version string from the current repository state.
131132

132133
To work in this image, replace ``poldracklab/fmriprep:latest`` with
133134
``fmriprep`` in any of the above commands.
134-
This image may be accessed by the `fmriprep-docker`_ wrapper via the
135-
``-i`` flag, e.g., ::
135+
This image may be accessed by the `Docker wrapper`_
136+
via the ``-i`` flag, e.g., ::
136137

137138
$ fmriprep-docker -i fmriprep --shell
138139

‎docs/docker.rst

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
.. include:: links.rst
2+
3+
.. _run_docker:
4+
5+
Running *fMRIPrep* via Docker containers
6+
========================================
7+
For every new version of *fMRIPrep* that is released, a corresponding Docker
8+
image is generated.
9+
The Docker image *becomes* a container when the execution engine loads the
10+
image and adds an extra layer that makes it *runnable*.
11+
In order to run *fMRIPrep* Docker images, the Docker Engine must be installed
12+
(see :ref:`installation_docker`).
13+
14+
If you used *fMRIPrep* via Docker in the past, you might need to pull down a
15+
more recent version of the image: ::
16+
17+
$ docker pull poldracklab/fmriprep:<latest-version>
18+
19+
You can run *fMRIPrep* interacting directly with the Docker Engine via the `docker run`
20+
command line, or you can use a lightweight wrapper we created for convenience.
21+
22+
.. _docker_wrapper:
23+
24+
Running *fMRIPrep* with the ``fmriprep-docker`` wrapper
25+
-------------------------------------------------------
26+
Before starting, make sure you
27+
`have the wrapper installed <installation.html#the-fmriprep-docker-wrapper>`__.
28+
When you run ``fmriprep-docker``, it will generate a Docker command line for you,
29+
print it out for reporting purposes, and then execute it without further action
30+
needed, e.g.::
31+
32+
$ fmriprep-docker /path/to/data/dir /path/to/output/dir participant
33+
RUNNING: docker run --rm -it -v /path/to/data/dir:/data:ro \
34+
-v /path/to_output/dir:/out poldracklab/fmriprep:1.0.0 \
35+
/data /out participant
36+
...
37+
38+
``fmriprep-docker`` accepts all of the typical options for *fMRIPrep* (see :ref:`usage`),
39+
automatically translating directories into Docker mount points.
40+
41+
We have published a `step-by-step tutorial
42+
<http://reproducibility.stanford.edu/fmriprep-tutorial-running-the-docker-image/>`_
43+
illustrating how to run ``fmriprep-docker``.
44+
This tutorial also provides valuable troubleshooting insights and advice on
45+
what to do after *fMRIPrep* has run.
46+
47+
48+
Running *fMRIPrep* directly interacting with the Docker Engine
49+
--------------------------------------------------------------
50+
If you need a finer control over the container execution, or you feel comfortable
51+
with the Docker Engine, avoiding the extra software layer of the wrapper might be
52+
a good decision.
53+
54+
**Accessing filesystems in the host within the container**.
55+
Containers are confined in a sandbox, so they can't access the host in any ways unless
56+
you explicitly prescribe acceptable accesses to the host.
57+
The Docker Engine provides mounting filesystems into the container with the ``-v`` argument
58+
and the following syntax: ``-v some/path/in/host:/absolute/path/within/container:ro``,
59+
where the trailing ``:ro`` specifies that the mount is read-only.
60+
The mount permissions modifiers can be omitted, which means the mount will have read-write
61+
permissions.
62+
In general, you'll want to at least provide two mount-points: one set in read-only mode
63+
for the input data and one read/write to store the outputs.
64+
Potentially, you'll want to provide one or two more mount-points: one for the working
65+
directory, in case you need to debug some issue or reuse pre-cached results; and
66+
a :ref:`TemplateFlow` folder to preempt the download of your favorite templates in every
67+
run.
68+
69+
**Running containers as a user**.
70+
By default, Docker will run the container as **root**.
71+
Some share systems my limit this feature and only allow running containers as a user.
72+
When the container is run as **root**, files written out to filesystems mounted from the
73+
host will have the user id ``1000`` by default.
74+
In other words, you'll need to be able to run as root in the host to change permissions
75+
or manage these files.
76+
Alternatively, running as a user allows preempting these permissions issues.
77+
It is possible to run as a user with the ``-u`` argument.
78+
In general, we will want to use the same user ID as the running user in the host to
79+
ensure the ownership of files written during the container execution.
80+
Therefore, you will generally run the container with ``-u $( id -u )``.
81+
82+
You may also invoke ``docker`` directly::
83+
84+
$ docker run -ti --rm \
85+
-v path/to/data:/data:ro \
86+
-v path/to/output:/out \
87+
poldracklab/fmriprep:<latest-version> \
88+
/data /out/out \
89+
participant
90+
91+
For example: ::
92+
93+
$ docker run -ti --rm \
94+
-v $HOME/ds005:/data:ro \
95+
-v $HOME/ds005/derivatives:/out \
96+
-v $HOME/tmp/ds005-workdir:/work \
97+
poldracklab/fmriprep:<latest-version> \
98+
/data /out/fmriprep-<latest-version> \
99+
participant \
100+
-w /work
101+
102+
Once the Docker Engine arguments are written, the remainder of the command line follows
103+
the :ref:`usage`.
104+
In other words, the first section of the command line is all equivalent to the ``fmriprep``
105+
executable in a bare-metal installation: ::
106+
107+
$ docker run -ti --rm \ | These lines
108+
-v $HOME/ds005:/data:ro \ | are equivalent
109+
-v $HOME/ds005/derivatives:/out \ | to the fmriprep
110+
-v $HOME/tmp/ds005-workdir:/work \ | executable.
111+
poldracklab/fmriprep:<latest-version> \ |
112+
\
113+
/data /out/fmriprep-<latest-version> \ | These lines
114+
participant \ | correspond to
115+
-w /work | fmriprep arguments.

‎docs/faq.rst

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ FAQ, Tips, and Tricks
66

77
Should I run quality control of my images before running *fMRIPrep*?
88
--------------------------------------------------------------------
9-
109
Yes. You should do so before any processing/analysis takes place.
1110

1211
Oftentimes (more often than we would like), images have fatal artifacts and problems.
@@ -27,7 +26,6 @@ some kind of preprocessing (see next question).
2726

2827
What if I find some images have undergone some pre-processing already (e.g., my T1w image is already skull-stripped)?
2928
---------------------------------------------------------------------------------------------------------------------
30-
3129
These images imply an unknown level of preprocessing (e.g. was it already bias-field corrected?),
3230
which makes it difficult to decide on best-practices for further processing.
3331
Hence, supporting such images was considered very low priority for *fMRIPrep*.
@@ -40,7 +38,6 @@ reverting to the original, defaced, T1w images to ensure more uniform preprocess
4038

4139
My *fMRIPrep* run is hanging...
4240
-------------------------------
43-
4441
When running on Linux platforms (or containerized environments, because they are built around
4542
Ubuntu), there is a Python bug that affects *fMRIPrep* that drives the Linux kernel to kill
4643
processes as a response to running out of memory.
@@ -62,7 +59,6 @@ Additionally, consider using the ``--low-mem`` flag, which will make some memory
6259

6360
ERROR: it appears that ``recon-all`` is already running
6461
-------------------------------------------------------
65-
6662
When running FreeSurfer_'s ``recon-all``, an error may say *it appears it is already running*.
6763
FreeSurfer creates files (called ``IsRunning.{rh,lh,lh+rh}``, under the ``scripts/`` folder)
6864
to determine whether it is already executing ``recon-all`` on that particular subject
@@ -117,7 +113,6 @@ In general, please be cautious of deleting files and mindful why a file may exis
117113

118114
Running subjects in parallel
119115
----------------------------
120-
121116
When running several subjects in parallel, and depending on your settings, fMRIPrep may
122117
fall into race conditions.
123118
A symptomatic output looks like: ::
@@ -155,7 +150,6 @@ number of CPUs divided by the number of subjects is most efficient.
155150

156151
A new version of *fMRIPrep* has been published, when should I upgrade?
157152
----------------------------------------------------------------------
158-
159153
We follow a philosophy of releasing very often, although the pace is slowing down
160154
with the maturation of the software.
161155
It is very likely that your version gets outdated over the extent of your study.
@@ -169,3 +163,16 @@ using the latest version of the tool.
169163
In any case, if you can find your release listed as *flagged* in `this file
170164
of our repo <https://github.com/poldracklab/fmriprep/blob/master/.versions.json>`__,
171165
then please update as soon as possible.
166+
167+
What is *TemplateFlow* for?
168+
---------------------------
169+
*TemplateFlow* enables *fMRIPrep* to generate preprocessed outputs spatially normalized to
170+
a number of different neuroimaging templates (e.g. MNI).
171+
For further details, please check `its documentation section <spaces.html#templateflow>`__.
172+
173+
I'm running *fMRIPrep* via Singularity containers - how can I troubleshoot problems?
174+
------------------------------------------------------------------------------------
175+
We have extended `this documentation <singularity.html>`__ to cover some of the most
176+
frequent issues other Singularity users have been faced with.
177+
Generally, users have found it hard to `get TemplateFlow and Singularity to work
178+
together <singularity.html#singularity-tf>`__.

‎docs/index.rst

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,15 @@ Contents
1515
:maxdepth: 3
1616

1717
installation
18-
changes
1918
usage
19+
docker
20+
singularity
2021
workflows
2122
sdc
22-
spaces
2323
outputs
24+
spaces
2425
faq
2526
contributors
2627
citing
2728
api/index.rst
29+
changes

0 commit comments

Comments
 (0)