Skip to content

Commit 05a8050

Browse files
ArshithaArshitha
committed
README update
1 parent 268cba5 commit 05a8050

File tree

1 file changed

+74
-32
lines changed

1 file changed

+74
-32
lines changed

‎README.md‎

Lines changed: 74 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -2,71 +2,113 @@
22

33
# DSST Defacing Pipeline
44

5-
The defacing pipeline for datasets curated by the [Data Science and Sharing Team (DSST)](https://cmn.nimh.nih.gov/dsst) are completed in four steps. Each of these steps is explained in more detail with an example in the next section. The pipeline requires a BIDS dataset as input.
5+
The DSST Defacing Pipeline has been developed to make the process of defacing anatomical scans of large datasets, visually inspecting for accuracy and fixing scans that fail visual inspection more efficient and straightforward. The pipeline _requires_ the input dataset to be in BIDS format. A conceptual description of the pipeline can found [here](#conceptual-design).
66

7-
1. Generate and finalize ["primary" scans](#glossary) to [other scans'](#glossary) mapping file.
8-
2. Deface primary scans
9-
with [@afni_refacer_run](https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/refacer/refacer_run.html) program
10-
developed by the AFNI Team. To deface remaining scans in the session, register them to the primary scan and use
11-
it's defacemask to generate a defaced image.
12-
**NOTE**: If a session doesn't have a T1w scan, then `@afni_refacer_run` is run on all every scan individually.
13-
3. Visually inspect defaced scans with your preferred QC tool.
14-
4. Fix defacings that failed visual inspection.
7+
## Usage Instructions
8+
### Clone this repository
9+
```bash
10+
git clone [email protected]:nih-fmrif/dsst-defacing-pipeline.git
11+
```
12+
### Run `dsst_defacing_wf.py`
13+
To deface anatomical scans in the dataset, run `dsst_defacing_wf.py` script.
14+
```
15+
% python src/dsst_defacing_wf.py -h
16+
usage: dsst_defacing_wf.py [-h] --input INPUT --output OUTPUT [--participant-id SUBJ_ID] [--session-id SESS_ID] [--no-clean]
17+
18+
Deface anatomical scans for a given BIDS dataset or a subject directory in BIDS format.
19+
20+
optional arguments:
21+
-h, --help show this help message and exit
22+
--input INPUT, -i INPUT
23+
Path to input BIDS dataset.
24+
--output OUTPUT, -o OUTPUT
25+
Path to output BIDS dataset with defaced scan.
26+
--participant-id SUBJ_ID, -p SUBJ_ID
27+
Subject ID associated with the participant. Since the input dataset is assumed to be BIDS valid, this argument expects subject IDs with 'sub-' prefix.
28+
--session-id SESS_ID, -s SESS_ID
29+
Session ID associated with the subject ID. If the BIDS input dataset contains sessions, then this argument expects session IDs with 'ses-' prefix.
30+
--no-clean If this argument is provided, then AFNI intermediate files are preserved.
1531
16-
![Generate and finalize "primary" scans to "secondary" scans mapping file.](images/pipeline_screen_quality.png)
32+
```
1733

18-
## Example
34+
The script can be run serially on a BIDS dataset or in parallel at subject/session level. The three methods of running the script have been described below with example commands:
1935

20-
### **Step 0:** Get data and code
21-
Clone this repository to a preferred location on your machine.
36+
#### Option 1: Serially
37+
If you have a small dataset with less than 10 subjects, then it might be easiest to run the defacing algorithm serially.
2238

2339
```bash
24-
git clone [email protected]:nih-fmrif/dsst-defacing-pipeline.git
40+
python dsst_defacing_wf.py -i <path/to/BIDS/input/dataset> -o <path/to/desired/output/directory>
2541
```
2642

27-
We'll be running the scripts on the [MyConnectome](https://openneuro.org/datasets/ds000031/versions/1.0.0) dataset. The dataset is available for download on OpenNeuro as [ds000031](https://openneuro.org/datasets/ds000031/versions/1.0.0/download).
43+
#### Option 2: In parallel at subject level
44+
If you have dataset with over 10 subjects, then it might be more practical to run the pipeline in parallel for every subject in the dataset using the `-p/--participant-id` option as follows:
2845

2946
```bash
30-
datalad install https://github.com/OpenNeuroDatasets/ds000031.git
47+
python dsst_defacing_wf.py -i <path/to/BIDS/input/dataset> -o <path/to/desired/output/directory> -p sub-<index>
3148
```
3249

33-
Download data in `anat` directories of the dataset.
50+
a. Assuming these scripts are run on the NIH HPC system, the first step would be to create a `swarm` file:
3451

35-
```bash
36-
datalad get sub-01/ses-*/anat
37-
```
52+
```bash
53+
for i in `ls -d <path/to/BIDS/input/dataset>*`; do \
54+
SUBJ=$(echo $i | sed 's|<path/to/BIDS/input/dataset>||g' ); \
55+
echo "python dsst_defacing_wf.py -i <path/to/BIDS/input/dataset> -o <path/to/desired/output/directory> -s $SUBJ"; \
56+
done > defacing_parallel_subject_level.swarm
57+
```
58+
Purpose: Loop through the dataset and find all subject directories to construct `dsst_defacing_wf.py` command with `-p/--participant-id` option.
3859

39-
### **Step 1:** Deface scans
40-
Run `dsst_defacing_wf.py` script that calls on `deface.py` and `register.py` to deface scans in the dataset.
60+
b. Run the swarm file with following command to start a swarm job
61+
```bash
62+
swarm -f .defacing_parallel_subject_level.swarm --module afni,fsl --merge-output --logdir swarm_log
63+
```
4164

42-
#### Option 1: Serially
43-
If you have a small dataset with less than 10 subjects, then it might be easiest to run the defacing algorithm serially.
65+
#### Option 3: In parallel at session level
66+
If the input dataset has multiple sessions per subject, then run the pipeline on every session in the dataset parallelly. Similar to Option 2, the following commands loop through the dataset to find subject and session IDs to create a `swarm` file to be run on NIH HPC systems.
4467

4568
```bash
46-
python dsst_defacing_wf.py -i ../datasets/ds000031 -o examples
69+
for i in `ls -d <path/to/BIDS/input/dataset>*`; do \
70+
SUBJ=$(echo $i | sed 's|<path/to/BIDS/input/dataset>||g' ); \
71+
echo "python dsst_defacing_wf.py -i <path/to/BIDS/input/dataset> -o <path/to/desired/output/directory> -s $SUBJ"; \
72+
done > defacing_parallel_subject_level.swarm
73+
```
74+
```bash
75+
swarm -f .defacing_parallel_subject_level.swarm --module afni,fsl --merge-output --logdir swarm_log
4776
```
4877

49-
#### Option 2: Parallelly
50-
If you have dataset with over 10 subjects, then it might be more practical to run it in parallel. Here's the command one would use to run it on NIH HPC:
78+
### Visually inspect defaced scans using VisualQC
79+
80+
Pre-requisite: Install VisualQC from https://raamana.github.io/visualqc/installation.html#stable-release[](https://raamana.github.io/visualqc/installation.html#stable-release)
5181

82+
Once VisualQC is installed, please run the following command to open VisualQC deface GUI to start visually inspecting defaced scans:
5283
```bash
53-
for i in `ls -d ../datasets/toy/*`; do SUBJ=$(echo $i | sed 's|../datasets/toy/||g' ); echo "python dsst_defacing_wf.py -i ../datasets/ds000031 -m examples/primary_to_others_mapping.json -o examples -s $SUBJ"; done > ./examples/defacing_parallel.swarm
54-
swarm -f ./examples/defacing_parallel.swarm --module afni,fsl --merge-output --logdir ./examples/swarm_log
84+
sh <path/to/desired/output/directory>/visualqc_prep/defacing_qc_cmd
5585
```
5686

57-
### **Step 2:** Visually QC defaced scans.
58-
5987
Visual QC defacing accuracy gallery https://raamana.github.io/visualqc/gallery_defacing.html
6088

61-
## Glossary
89+
90+
## Terminology
91+
While describing the process, we frequently use the following terms:
6292

6393
- **Primary Scan:** The best quality T1w scan within a session. For programmatic selection, we assume that the most
6494
recently acquired T1w scan is of the best quality.
6595
- **Other/Secondary Scans:** All scans *except* the primary scan are grouped together and referred to as "other" or "
6696
secondary" scans for a given session.
97+
- **Mapping File:** A JSON file that assigns maps a primary scan (or `primary_t1`) to all other scans within a session. Please find an example file [here]().
6798
- **[VisualQC](https://raamana.github.io/visualqc):** A suite of QC tools developed by Pradeep Raamana (Assistant
6899
Professor at University of Pittsburgh).
69100

101+
## Conceptual design
102+
1. Generate a ["primary" scans](#terminology) to [other scans'](#terminology) mapping file.
103+
2. Deface primary scans
104+
with [@afni_refacer_run](https://afni.nimh.nih.gov/pub/dist/doc/htmldoc/tutorials/refacer/refacer_run.html) program
105+
developed by the AFNI Team.
106+
3. To deface remaining scans in the session, register them to the primary scan (using FSL `flirt` command) and then use the primary scan's defacemask to generate a defaced image (using `fslmaths` command).
107+
4. Visually inspect defaced scans with [VisualQC](https://raamana.github.io/visualqc) deface tool or any other preferred tool.
108+
5. Correct/fix defaced scans that failed visual inspection. See [here]() for more info on types of failures.
109+
110+
![Defacing Pipeline flowchart](images/pipeline_screen_quality.png)
111+
70112
## References
71113

72114
1. Theyers AE, Zamyadi M, O'Reilly M, Bartha R, Symons S, MacQueen GM, Hassel S, Lerch JP, Anagnostou E, Lam RW, Frey

0 commit comments

Comments
 (0)