Skip to content

Commit 0ca95f1

Browse files
yarikopticyarikoptic
committed
Merge tag 'v0.5' into debian
New upstream release The first release after major refactoring: - Refactored into a proper `heudiconv` Python module - `heuristics` is now a `heudiconv.heuristics` submodule - you can specify shipped heuristics by name (e.g. `-f reproin`) without providing full path to their files - you need to use `--files` (not just positional argument(s)) if not using `--dicom_dir_templates` or `--subjects` to point to data files or directories with input DICOMs - `Dockerfile` is generated by [neurodocker](https://github.com/kaczmarj/neurodocker) - Logging verbosity reduced - Increased leniency with missing DICOM fields - `dbic_bids` heuristic renamed into reproin - [LICENSE](https://github.com/nipy/heudiconv/blob/master/LICENSE) with Apache 2.0 license for the project - [CHANGELOG.md](https://github.com/nipy/heudiconv/blob/master/CHANGELOG.md) - [Regression testing](https://github.com/nipy/heudiconv/blob/master/tests/test_regression.py) on real data (using datalad) - A dedicated [ReproIn](https://github.com/repronim/reproin) project with details about ReproIn setup/specification and operation using `reproin` heuristic shipped with heudiconv - [utils/test-compare-two-versions.sh](utils/test-compare-two-versions.sh) helper to compare conversions with two different versions of heudiconv - Support for converters other than `dcm2niix`, which is now the default. Explicitly specify `-c none` to only prepare conversion specification files without performing actual conversion - Compatibility with Nipype 1.0, PyDicom 1.0, and upcoming DataLad 0.10 - Consistency with converted files permissions - Ensured subject id for BIDS conversions will be BIDS compliant - Re-add `seqinfo` fields as column names in generated `dicominfo` - More robust sanity check of the regex reformatted .json file to avoid numeric precision issues - Many other various issues * tag 'v0.5': (187 commits) DOC: Extended CHANGELOG.md a bit DOC: Point to the apache 2.0 license ENH: just add a warning whenever we buttplug for dcmstack BF(PY3): in python3 message has quotes around name, so made it easier check BF(workaround): provide explicit output_dir (to be = tmpdir which is workdir) for dcm2niix call reraise is import of dcmstack is failing for other natural causes RF: make use of external.dcmstack and delay its import until use BF(nasty): for dicom->pydicom - feed pydicom as dicom into sys.modules Make calls to import pydicom compatible with pydicom 1.0 BF+RF: it is files, not paths name of the arg. Pass as an arg not kwarg minor pep8ing BF: use ds.repo.set_metadata (not ds.metadata) - compat with datalad 0.10 RF: Move mark_sensitive into dlad.py module Create LICENSE ENH: reincarnate the paranoid test of a json tuned up with regular expressions Allow to override locator from the cmdline Fixed spaces problem, to make some lines execute _inside_ loop, rather that at the end of the loop BF: make the comparator work with both versions specifications of heuristics BF: the point of "ls" to list so we should print the item Simplify dicom check, add more comments for test_monitor ...
2 parents e0e6b9d + 80a22ce commit 0ca95f1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+3517
-2571
lines changed

‎.coveragerc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
[run]
22
include = tests/*
3-
bin/*
3+
heudiconv/*
44
setup.py

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
.cache/
33
.coverage
44
*.egg-info/
5+
.idea/

‎.mailmap

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Mathias Goncalves <[email protected]> <[email protected]>
2+
Matteo Visconti di Oleggio Castello <[email protected]>
3+
Matteo Visconti di Oleggio Castello <[email protected]>
4+
Matteo Visconti di Oleggio Castello <[email protected]>
5+
Chris Filo Gorgolewski <[email protected]> Chris Gorgolewski <[email protected]>

‎.travis.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ before_install:
2727
- source venv/bin/activate
2828
- python --version # just to check
2929
- pip install -r dev-requirements.txt
30+
- pip install requests # below installs pyld but that assumes we have requests already
3031
- pip install datalad
3132
- pip install codecov pytest
3233

@@ -35,7 +36,7 @@ install:
3536
- git config --global user.name "Travis Almighty"
3637

3738
script:
38-
- coverage run `which py.test` -s -v tests heuristics
39+
- coverage run `which py.test` -s -v tests heudiconv/heuristics/*.py
3940

4041
after_success:
4142
- codecov

‎CHANGELOG.md

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Changelog
2+
All notable changes to this project will be documented (for humans) in this file.
3+
4+
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
5+
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
6+
7+
## [0.5] - 2018-03-01
8+
The first release after major refactoring:
9+
10+
### Changed
11+
- Refactored into a proper `heudiconv` Python module
12+
- `heuristics` is now a `heudiconv.heuristics` submodule
13+
- you can specify shipped heuristics by name (e.g. `-f reproin`)
14+
without providing full path to their files
15+
- you need to use `--files` (not just positional argument(s)) if not
16+
using `--dicom_dir_templates` or `--subjects` to point to data files
17+
or directories with input DICOMs
18+
- `Dockerfile` is generated by [neurodocker](https://github.com/kaczmarj/neurodocker)
19+
- Logging verbosity reduced
20+
- Increased leniency with missing DICOM fields
21+
- `dbic_bids` heuristic renamed into reproin
22+
### Added
23+
- [LICENSE](https://github.com/nipy/heudiconv/blob/master/LICENSE)
24+
with Apache 2.0 license for the project
25+
- [CHANGELOG.md](https://github.com/nipy/heudiconv/blob/master/CHANGELOG.md)
26+
- [Regression testing](https://github.com/nipy/heudiconv/blob/master/tests/test_regression.py) on real data (using datalad)
27+
- A dedicated [ReproIn](https://github.com/repronim/reproin) project
28+
with details about ReproIn setup/specification and operation using
29+
`reproin` heuristic shipped with heudiconv
30+
- [utils/test-compare-two-versions.sh](utils/test-compare-two-versions.sh)
31+
helper to compare conversions with two different versions of heudiconv
32+
### Removed
33+
- Support for converters other than `dcm2niix`, which is now the default.
34+
Explicitly specify `-c none` to only prepare conversion specification
35+
files without performing actual conversion
36+
### Fixed
37+
- Compatibility with Nipype 1.0, PyDicom 1.0, and upcoming DataLad 0.10
38+
- Consistency with converted files permissions
39+
- Ensured subject id for BIDS conversions will be BIDS compliant
40+
- Re-add `seqinfo` fields as column names in generated `dicominfo`
41+
- More robust sanity check of the regex reformatted .json file to avoid
42+
numeric precision issues
43+
- Many other various issues
44+
45+
## [0.4] - 2017-10-15
46+
A usable release to support [DBIC] use-case
47+
### Added
48+
- more testing
49+
### Changes
50+
- Dockerfile updates (added pigz, progressed forward [dcm2niix])
51+
### Fixed
52+
- correct date/time in BIDS `_scans` files
53+
- sort entries in `_scans` by date and then filename
54+
55+
## [0.3] - 2017-07-10
56+
A somewhat working release on the way to support [DBIC] use-case
57+
### Added
58+
- more tests
59+
- groupping of dicoms by series if provided
60+
- many more features and fixes
61+
62+
## [0.2] - 2016-10-20
63+
An initial release on the way to support [DBIC] use-case
64+
### Added
65+
- basic Python project assets (`setup.py`, etc)
66+
- basic tests
67+
- [datalad] support
68+
- dbic_bids heuristic
69+
- `--dbg` command line flag to enter `pdb` environment upon failure
70+
## Fixed
71+
- Better Python3 support
72+
- Better PEP8 compliance
73+
74+
## [0.1] - 2015-09-23
75+
76+
Initial version
77+
78+
---
79+
80+
Just a template for future records:
81+
82+
## [Unreleased] - Date
83+
TODO Summary
84+
### Added
85+
### Changed
86+
### Deprecated
87+
### Fixed
88+
### Removed
89+
### Security
90+
91+
---
92+
93+
## References
94+
[DBIC]: http://dbic.dartmouth.edu
95+
[datalad]: http://datalad.org
96+
[dcm2niix]: https://github.com/rordenlab/dcm2niix

‎Dockerfile

Lines changed: 163 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,164 @@
1+
# Generated by Neurodocker v0.3.2.
2+
#
3+
# Thank you for using Neurodocker. If you discover any issues
4+
# or ways to improve this software, please submit an issue or
5+
# pull request on our GitHub repository:
6+
# https://github.com/kaczmarj/neurodocker
7+
#
8+
# Timestamp: 2018-02-02 16:32:55
19

2-
FROM continuumio/miniconda
3-
4-
MAINTAINER <[email protected]>
5-
6-
RUN apt-get update && apt-get upgrade -y && \
7-
apt-get install -y g++ pkg-config make && \
8-
apt-get clean -y && apt-get autoclean -y && apt-get autoremove -y
9-
RUN (wget -O- http://neuro.debian.net/lists/jessie.us-nh.full | tee /etc/apt/sources.list.d/neurodebian.sources.list) && \
10-
apt-key adv --recv-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 && \
11-
apt-get update -qq && apt-get install -y git-annex-standalone && \
12-
apt-get clean -y && apt-get autoclean -y && apt-get autoremove -y
13-
RUN conda install -y -c conda-forge nipype && \
14-
conda install cmake && \
15-
pip install https://github.com/moloney/dcmstack/archive/c12d27d2c802d75a33ad70110124500a83e851ee.zip && \
16-
pip install datalad && \
17-
conda clean -tipsy && rm -rf ~/.pip/
18-
RUN apt-get update && apt-get upgrade -y && \
19-
apt-get install -y pigz && \
20-
apt-get clean -y && apt-get autoclean -y && apt-get autoremove -y && \
21-
cd /tmp && git clone https://github.com/neurolabusc/dcm2niix.git && \
22-
cd dcm2niix && \
23-
git checkout 6ba27b9befcbae925209664bb8acbb00e266114a && \
24-
mkdir build && cd build && cmake -DBATCH_VERSION=ON .. && \
25-
make && make install && \
26-
cd / && rm -rf /tmp/dcm2niix
27-
28-
COPY bin/heudiconv /usr/local/bin/heudiconv
29-
RUN chmod +x /usr/local/bin/heudiconv
30-
RUN mkdir /heuristics
31-
COPY heuristics/convertall.py /heuristics
32-
RUN chmod +x /heuristics/convertall.py
33-
RUN git config --global user.email "[email protected]" && \
34-
git config --global user.name "Docker Almighty"
35-
36-
ENTRYPOINT ["/usr/local/bin/heudiconv"]
10+
FROM debian:stretch
11+
12+
ARG DEBIAN_FRONTEND=noninteractive
13+
14+
#----------------------------------------------------------
15+
# Install common dependencies and create default entrypoint
16+
#----------------------------------------------------------
17+
ENV LANG="en_US.UTF-8" \
18+
LC_ALL="C.UTF-8" \
19+
ND_ENTRYPOINT="/neurodocker/startup.sh"
20+
RUN apt-get update -qq && apt-get install -yq --no-install-recommends \
21+
apt-utils bzip2 ca-certificates curl locales unzip \
22+
&& apt-get clean \
23+
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
24+
&& localedef --force --inputfile=en_US --charmap=UTF-8 C.UTF-8 \
25+
&& chmod 777 /opt && chmod a+s /opt \
26+
&& mkdir -p /neurodocker \
27+
&& if [ ! -f "$ND_ENTRYPOINT" ]; then \
28+
echo '#!/usr/bin/env bash' >> $ND_ENTRYPOINT \
29+
&& echo 'set +x' >> $ND_ENTRYPOINT \
30+
&& echo 'if [ -z "$*" ]; then /usr/bin/env bash; else $*; fi' >> $ND_ENTRYPOINT; \
31+
fi \
32+
&& chmod -R 777 /neurodocker && chmod a+s /neurodocker
33+
ENTRYPOINT ["/neurodocker/startup.sh"]
34+
35+
RUN apt-get update -qq \
36+
&& apt-get install -y -q --no-install-recommends git \
37+
gcc \
38+
pigz \
39+
&& apt-get clean \
40+
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
41+
42+
#------------------------
43+
# Install dcm2niix v1.0.20171215
44+
#------------------------
45+
WORKDIR /tmp
46+
RUN deps='cmake g++ gcc git make pigz zlib1g-dev' \
47+
&& apt-get update -qq && apt-get install -yq --no-install-recommends $deps \
48+
&& apt-get clean \
49+
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
50+
&& mkdir dcm2niix \
51+
&& curl -sSL https://github.com/rordenlab/dcm2niix/tarball/v1.0.20171215 | tar xz -C dcm2niix --strip-components 1 \
52+
&& mkdir dcm2niix/build && cd dcm2niix/build \
53+
&& cmake .. && make \
54+
&& make install \
55+
&& rm -rf /tmp/*
56+
57+
COPY [".", "/src/heudiconv"]
58+
59+
#------------------
60+
# Install Miniconda
61+
#------------------
62+
ENV CONDA_DIR=/opt/conda \
63+
PATH=/opt/conda/bin:$PATH
64+
RUN echo "Downloading Miniconda installer ..." \
65+
&& miniconda_installer=/tmp/miniconda.sh \
66+
&& curl -sSL --retry 5 -o $miniconda_installer https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh \
67+
&& /bin/bash $miniconda_installer -b -p $CONDA_DIR \
68+
&& rm -f $miniconda_installer \
69+
&& conda config --system --prepend channels conda-forge \
70+
&& conda config --system --set auto_update_conda false \
71+
&& conda config --system --set show_channel_urls true \
72+
&& conda clean -tipsy && sync
73+
74+
#-------------------------
75+
# Create conda environment
76+
#-------------------------
77+
RUN conda create -y -q --name neuro python=2 \
78+
traits=4.6.0 \
79+
&& sync && conda clean -tipsy && sync \
80+
&& /bin/bash -c "source activate neuro \
81+
&& pip install -q --no-cache-dir https://github.com/moloney/dcmstack/tarball/master \
82+
/src/heudiconv[all]" \
83+
&& sync \
84+
&& sed -i '$isource activate neuro' $ND_ENTRYPOINT
85+
86+
#--------------------------------------------------
87+
# Add NeuroDebian repository
88+
# Please note that some packages downloaded through
89+
# NeuroDebian may have restrictive licenses.
90+
#--------------------------------------------------
91+
RUN apt-get update -qq && apt-get install -yq --no-install-recommends dirmngr gnupg \
92+
&& apt-get clean \
93+
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
94+
&& curl -sSL http://neuro.debian.net/lists/stretch.us-nh.full \
95+
> /etc/apt/sources.list.d/neurodebian.sources.list \
96+
&& curl -sSL https://dl.dropbox.com/s/zxs209o955q6vkg/neurodebian.gpg \
97+
| apt-key add - \
98+
&& (apt-key adv --refresh-keys --keyserver hkp://pool.sks-keyservers.net:80 0xA5D32F012649A5A9 || true) \
99+
&& apt-get update
100+
101+
# Install NeuroDebian packages
102+
RUN apt-get update -qq && apt-get install -yq --no-install-recommends git-annex-standalone \
103+
&& apt-get clean \
104+
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
105+
106+
ENTRYPOINT ["/neurodocker/startup.sh", "heudiconv"]
107+
108+
#--------------------------------------
109+
# Save container specifications to JSON
110+
#--------------------------------------
111+
RUN echo '{ \
112+
\n "pkg_manager": "apt", \
113+
\n "check_urls": true, \
114+
\n "instructions": [ \
115+
\n [ \
116+
\n "base", \
117+
\n "debian:stretch" \
118+
\n ], \
119+
\n [ \
120+
\n "install", \
121+
\n [ \
122+
\n "git", \
123+
\n "gcc", \
124+
\n "pigz" \
125+
\n ] \
126+
\n ], \
127+
\n [ \
128+
\n "dcm2niix", \
129+
\n { \
130+
\n "version": "v1.0.20171215" \
131+
\n } \
132+
\n ], \
133+
\n [ \
134+
\n "copy", \
135+
\n [ \
136+
\n ".", \
137+
\n "/src/heudiconv" \
138+
\n ] \
139+
\n ], \
140+
\n [ \
141+
\n "miniconda", \
142+
\n { \
143+
\n "env_name": "neuro", \
144+
\n "conda_install": "python=2 traits=4.6.0", \
145+
\n "pip_install": "https://github.com/moloney/dcmstack/tarball/master /src/heudiconv[all]", \
146+
\n "activate": true \
147+
\n } \
148+
\n ], \
149+
\n [ \
150+
\n "neurodebian", \
151+
\n { \
152+
\n "os_codename": "stretch", \
153+
\n "download_server": "usa-nh", \
154+
\n "pkgs": "git-annex-standalone" \
155+
\n } \
156+
\n ], \
157+
\n [ \
158+
\n "entrypoint", \
159+
\n "/neurodocker/startup.sh heudiconv" \
160+
\n ] \
161+
\n ], \
162+
\n "generation_timestamp": "2018-02-02 16:32:55", \
163+
\n "neurodocker_version": "0.3.2" \
164+
\n}' > /neurodocker/neurodocker_specs.json

‎LICENSE

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
Copyright [2014-2018] [Heudiconv developers]
2+
3+
Licensed under the Apache License, Version 2.0 (the "License");
4+
you may not use this file except in compliance with the License.
5+
You may obtain a copy of the License at
6+
7+
http://www.apache.org/licenses/LICENSE-2.0
8+
9+
Unless required by applicable law or agreed to in writing, software
10+
distributed under the License is distributed on an "AS IS" BASIS,
11+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
See the License for the specific language governing permissions and
13+
limitations under the License.

‎Makefile

Lines changed: 0 additions & 15 deletions
This file was deleted.

0 commit comments

Comments
 (0)