Skip to content

Commit 48e5e15

Browse files
oestebanoesteban
committed
ENH: Port *fMRIPrep*'s developer environment guidelines
1 parent 0375f5d commit 48e5e15

File tree

2 files changed

+174
-0
lines changed

2 files changed

+174
-0
lines changed

‎docs/devs/devenv.md

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
2+
This document explains how to prepare a new development environment and update an existing environment, as necessary, for the development of *NiPreps*' components.
3+
Some components may deviate from these guidelines, in such a case, please follow the guidelines provided in their documentation.
4+
5+
If you plan to contribute back to the community, making your code available via pull-request, please make sure to have read and understood the [Community Documents and Contributor Guidelines](../community/index.md).
6+
If you plan to distribute derived code, please follow our [licensing guidelines](../community/licensing.md).
7+
8+
Development in Docker is encouraged, for the sake of consistency and portability.
9+
By default, work should be built off of [`nipreps/fmriprep:unstable`](https://hub.docker.com/r/nipreps/fmriprep/), which tracks the `master` branch, or `nipreps/fmriprep:latest`, which tracks the latest release version (see [BIDS-Apps execution guide](../apps/docker.md) for the basic procedure for running).
10+
11+
It will be assumed the developer has a working repository in `$HOME/projects/fmriprep`, and examples are also given for [niworkflows](https://github.com/nipreps/niworkflows) and *NiPype*.
12+
13+
## Patching a working copy into a Docker container
14+
15+
In order to test new code without rebuilding the Docker image, it is possible to mount working repositories as source directories within the container.
16+
The [Docker wrapper](../apps/docker.md#running-a-niprep-with-a-lightweight-wrapper) script simplifies this for the most common repositories:
17+
18+
```shell
19+
-f PATH, --patch-fmriprep PATH
20+
working fmriprep repository (default: None)
21+
-n PATH, --patch-niworkflows PATH
22+
working niworkflows repository (default: None)
23+
-p PATH, --patch-nipype PATH
24+
working nipype repository (default: None)
25+
```
26+
27+
For instance, if your repositories are contained in `$HOME/projects`:
28+
29+
```shell
30+
$ fmriprep-docker -f $HOME/projects/fmriprep/fmriprep \
31+
-n $HOME/projects/niworkflows/niworkflows \
32+
-p $HOME/projects/nipype/nipype \
33+
-i nipreps/fmriprep:latest \
34+
$HOME/fullds005 $HOME/dockerout participant
35+
```
36+
37+
Note the `-i` flag allows you to specify an image.
38+
39+
When invoking `docker` directly, the mount options must be specified
40+
with the `-v` flag:
41+
42+
```shell
43+
-v $HOME/projects/fmriprep/fmriprep:/usr/local/miniconda/lib/python3.7/site-packages/fmriprep:ro
44+
-v $HOME/projects/niworkflows/niworkflows:/usr/local/miniconda/lib/python3.7/site-packages/niworkflows:ro
45+
-v $HOME/projects/nipype/nipype:/usr/local/miniconda/lib/python3.7/site-packages/nipype:ro
46+
```
47+
48+
For example,
49+
50+
```shell
51+
$ docker run --rm -v $HOME/ds005:/data:ro -v $HOME/dockerout:/out \
52+
-v $HOME/projects/fmriprep/fmriprep:/usr/local/miniconda/lib/python3.7/site-packages/fmriprep:ro \
53+
nipreps/fmriprep:latest /data /out/out participant \
54+
-w /out/work/
55+
```
56+
57+
In order to work directly in the container, pass the `--shell` flag to `fmriprep-docker`
58+
59+
```shell
60+
$ fmriprep-docker --shell $HOME/ds005 $HOME/dockerout participant
61+
```
62+
63+
This is the equivalent of using `--entrypoint=bash` and omitting the *fMRIPrep* arguments in a `docker` command:
64+
65+
```shell
66+
$ docker run --rm -v $HOME/ds005:/data:ro -v $HOME/dockerout:/out \
67+
-v $HOME/projects/fmriprep/fmriprep:/usr/local/miniconda/lib/python3.7/site-packages/fmriprep:ro --entrypoint=bash \
68+
nipreps/fmriprep:latest
69+
```
70+
71+
Patching containers can be achieved in Singularity analogous to `docker` using the `--bind` (`-B`) option:
72+
73+
```shell
74+
$ singularity run \
75+
-B $HOME/projects/fmriprep/fmriprep:/usr/local/miniconda/lib/python3.7/site-packages/fmriprep \
76+
fmriprep.img \
77+
/scratch/dataset /scratch/out participant -w /out/work/
78+
```
79+
80+
## Adding dependencies
81+
82+
New dependencies to be inserted into the Docker image will either be Python or non-Python dependencies.
83+
Python dependencies may be added in three places, depending on whether the package is large or non-release versions are required.
84+
The image [must be rebuilt](#rebuilding-docker-image) after any dependency changes.
85+
86+
Python dependencies should generally be included in the appropriate dependency metadata of the `setup.cfg` file found at the root of each repository.
87+
If some the dependency must be a particular version (or set thereof), it is possible to use version filters in this `setup.cfg` file.
88+
89+
For large Python dependencies where there will be a benefit to pre-compiled binaries, [conda](https://github.com/conda/conda) packages may also be added to the `conda install` line in the [Dockerfile](https://github.com/nipreps/fmriprep/blob/29133e5e9f92aae4b23dd897f9733885a60be311/Dockerfile#L46).
90+
91+
Non-Python dependencies must also be installed in the Dockerfile, via a `RUN` command.
92+
For example, installing an `apt` package may be done as follows:
93+
94+
```Dockerfile
95+
RUN apt-get update && \
96+
apt-get install -y <PACKAGE>
97+
```
98+
99+
## (Re)Building Docker image
100+
101+
If it is necessary to (re)build the Docker image, a local image named `fmriprep` may be built from within the local repository.
102+
Let's assume it is located in `~/projects/fmriprep`:
103+
104+
```shell
105+
~/projects/fmriprep$ VERSION=$( python get_version.py )
106+
~/projects/fmriprep$ docker build -t fmriprep --build-arg VERSION=$VERSION .
107+
```
108+
109+
The `VERSION` build argument is necessary to ensure that help text can be reliably generated.
110+
The `get_version.py` tool constructs the version string from the current repository state.
111+
112+
To work in this image, replace `nipreps/fmriprep:latest` with just `fmriprep` in any of the above commands.
113+
This image may be accessed by the [Docker wrapper](../apps/docker.md#running-a-niprep-with-a-lightweight-wrapper) via the `-i` flag, e.g.:
114+
115+
```shell
116+
$ fmriprep-docker -i fmriprep --shell
117+
```
118+
119+
## Code-Server Development Environment (Experimental)
120+
121+
To get the best of working with containers and having an interactive development environment, we have an experimental setup with [code-server](https://github.com/cdr/code-server).
122+
123+
!!! important
124+
125+
We have [a video walking through the process](https://youtu.be/bkZ-NyUaTvg) if you want a visual guide.
126+
127+
128+
**1. Build the Docker image**.
129+
We will use the `Dockerfile_devel` file to build our development docker image:
130+
```shell
131+
$ cd $HOME/projects/fmriprep
132+
$ docker build -t fmriprep_devel -f Dockerfile_devel .
133+
```
134+
135+
**2. Run the Docker image**
136+
We can start a docker container using the image we built (`fmriprep_devel`):
137+
```shell
138+
$ docker run -it -p 127.0.0.1:8445:8080 -v ${PWD}:/src/fmriprep fmriprep_devel:latest
139+
```
140+
141+
!!! important "Windows Users"
142+
143+
If you are using windows shell, `${PWD}` may not be defined, instead use the absolute path to your repository.
144+
145+
!!! important "Docker-Toolbox"
146+
147+
If you are using Docker-Toolbox, you will need to change your virtualbox settings using [these steps as a guide](https://github.com/jdkent/tutDockerRstudio#additional-setup-for-docker-toolbox). For step 6, instead of `Name = rstudio; Host Port = 8787; Guest Port = 8787`, have `Name = code-server; Host Port = 8443; Guest Port = 8080`. Then in the docker command above, change `127.0.0.1:8445:8080` to `192.168.99.100:8445:8080`.
148+
149+
If the container started correctly, you should see the following on your console:
150+
151+
```shell
152+
INFO Server listening on http://localhost:8080
153+
INFO - No authentication
154+
INFO - Not serving HTTPS
155+
```
156+
157+
Now you can switch to your favorite browser and go to: [127.0.0.1:8445](http://127.0.0.1:8445) (or [192.168.99.100:8445](http://192.168.99.100:8445) for Docker Toolbox).
158+
159+
**3. Copy `fmriprep.egg-info` into your `fmriprep/` project directory**
160+
`fmriprep.egg-info` makes the package exacutable inside the docker container.
161+
Open a terminal in vscode and type the following:
162+
163+
```shell
164+
$ cp -R /src/fmriprep.egg-info /src/fmriprep/
165+
```
166+
167+
### Code-Server Development Environment Features
168+
- The editor is [vscode](https://code.visualstudio.com/docs)
169+
170+
- There are several preconfigured debugging tests under the debugging icon in the activity bar
171+
- see [vscode debugging python](https://code.visualstudio.com/docs/python/debugging) for details.
172+
173+
- The `gitlens` and `python` extensions are preinstalled to improve the development experience in vscode.

‎mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ nav:
2727
- Code of Conduct: community/CODE_OF_CONDUCT.md
2828
- Licensing: community/licensing.md
2929
- Developers:
30+
- Developer Environment: devs/devenv.md
3031
- Versions Matrix: devs/versions.md
3132
- Releases: devs/releases.md
3233
theme:

0 commit comments

Comments
 (0)