Merge pull request #237 from FCP-INDI/feature/acpc

sgiavasis · web-flow · commit 78e810f79b36 · 2020-09-22T15:59:01.000-04:00
Feature/acpc
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -4,7 +4,7 @@ commands:
     steps:
       - run:
           name: 🔧 Setting version
-          command: if [ -f /build/build_version.txt ]; then echo "export BUILD_VERSION=$(cat /build/build_version.txt)" >> $BASH_ENV; else echo "export BUILD_VERSION=nightly" >> $BASH_ENV; fi
+          command: if [[ $CIRCLE_TAG =~ v(.+)(-source) ]]; then echo "export BUILD_VERSION=v${BASH_REMATCH[1]}" >> $BASH_ENV; else echo "export BUILD_VERSION=nightly" >> $BASH_ENV; fi
   install-build-dependencies:
     parameters:
       version:
@@ -74,7 +74,6 @@ commands:
             git rm -fr docs/.doctrees || rm -fr docs/.doctrees || true
             git rm -fr docs/**/.doctrees || rm -fr docs/**/.doctrees || true
             git rm -fr bin || rm -fr bin || true
-            git rm -fr build_version.txt || rm -fr build_version.txt || true
             git rm -fr C-PAC || rm -fr C-PAC || true
             git rm -fr cpac || rm -fr cpac || true
             git rm -fr scripts/__pycache__ || rm -fr scripts/__pycache__ || true
@@ -92,7 +91,6 @@ commands:
           root: /
           paths: 
             - build/404.html
-            - build/build_version.txt
             - build/docs
             - build/index.html
             - build/scripts
@@ -121,9 +119,6 @@ jobs:
       - image: python:3.7
     steps:
       - checkout
-      - run:
-          name: 🔬 Checking for tag file
-          command: test -f ./build_version.txt
       - get-version
       - install-build-dependencies:
           version: ${BUILD_VERSION}
@@ -170,24 +165,24 @@ workflows:
       - build-nightly:
           filters:
             branches:
-              only:
-                - source
+              only: source
       - build-version:
           filters:
+            tags:
+              only: /^v.*(-source)$/
             branches:
-              only:
-                - build-version
+              ignore: /.*/
       - deploy-nightly:
           filters:
             branches:
-              only:
-                - source
+              only: source
           requires:
             - build-nightly
       - deploy-version:
           filters:
+            tags:
+              only: /^v.*(-source)$/
             branches:
-              only:
-                - build-version
+              ignore: /.*/
           requires:
             - build-version
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -9,18 +9,16 @@ Please, always base changes on the `source` branch. `master` branch will be over
 Pushes to `source` will rebuild docs at https://fcp-indi.github.io/docs/nightly
 
 #### C-PAC release tags
-Pushes to `build-version` will look for a C-PAC version tag (e.g, `v1.7.0`) in a file called `build_version.txt` at the root of the repo.
+Tags in the format `v.*-source` will build docs for the given version.
 
-If that file exists and includes only an existing tag, docs should build for the given version.
-
-If `build_version.txt` does not exist or the included tag doesn't exist in https://github.com/FCP-INDI/C-PAC/tags, the build should fail.
+If a matching version tag doesn't exist in https://github.com/FCP-INDI/C-PAC/tags, the build should fail.
 
 Steps to build a release:
 1. Checkout a commit from the `source` from a time appropriate for documentation for the branch being built/rebuilt.
-2. Name your branch (`git switch -c` or `git checkout -b`) either `build-version` or something more specific
-3. Add a text file `build_version.txt` at the root of the repository in your new branch. This text file should include the version tag for the version of C-PAC and nothing else.
-4. Make any changes you need for the specific version.
-5. (Force) push to the remote branch `build-version`.
+2. If you need to make changes, create a branch (`git switch -c` or `git checkout -b`).
+3. Make any changes you need for the specific version.
+4. Tag the commit that's ready to build (`git tag v`version`-source`)
+5. (Force) push to the remote tag (`git push origin v`version`-source`).
 6. CircleCI should deploy the versioned documentation. If the version tag is the newest, it should also overwrite `latest` with these documents. 
 
 ## Guidelines
diff --git a/docs/_sources/_images/acpc_gui.png b/docs/_sources/_images/acpc_gui.png
diff --git a/docs/_sources/_images/anat_init_options.png b/docs/_sources/_images/anat_init_options.png
diff --git a/docs/_sources/_static/flowcharts/anatomical.svg b/docs/_sources/_static/flowcharts/anatomical.svg
diff --git a/docs/_sources/_static/flowcharts/pipeline-individual.svg b/docs/_sources/_static/flowcharts/pipeline-individual.svg
diff --git a/docs/_sources/conf.py b/docs/_sources/conf.py
@@ -183,13 +183,9 @@ def _gh_rate_limit():
     gh_tags = []
 gh_tags.sort(reverse=True)
 
-build_version_path = os.path.abspath(os.path.join(
-    __file__, os.pardir, os.pardir, os.pardir, 'build_version.txt'
-))
 # don't build release notes for newer releases
-if os.path.exists(build_version_path):
-    with open(build_version_path, 'r') as bvf:
-        build_version = bvf.read().strip()
+build_version = os.environ.get('CIRCLE_TAG', '').rstrip('-source')
+if len(build_version):
     gh_tags = [gh_tag for gh_tag in gh_tags if compare_versions(
         build_version, gh_tag
     )]
diff --git a/docs/_sources/user/anat.rst b/docs/_sources/user/anat.rst
@@ -3,15 +3,16 @@
 
 .. raw:: html
 
-    <div class="flowchart-container"><object data="../../_static/flowcharts/anatomical.svg" type="image/svg+xml"></object></div>
+    <div class="flowchart-container"><object data="../_static/flowcharts/anatomical.svg" type="image/svg+xml"></object></div>
 
 Initial Preprocessing
 ^^^^^^^^^^^^^^^^^^^^^
 
-Initial preprocessing offers methods like `non-local means filtering <https://www.iro.umontreal.ca/~mignotte/IFT6150/Articles/Buades-NonLocal.pdf>`_ and `N4 bias field correction <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3071855/>`_ to preprocess anatomical images.
+Initial preprocessing offers methods like `ACPC Alignment <https://doi.org/10.1016/j.neuroimage.2013.04.127>`_ , `non-local means filtering <https://www.iro.umontreal.ca/~mignotte/IFT6150/Articles/Buades-NonLocal.pdf>`_ and `N4 bias field correction <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3071855/>`_ to preprocess anatomical images.
 
 C-PAC provides options for configuring initial preprocessing - users can select:
 
+* `ACPC Alignment <https://github.com/Washington-University/HCPpipelines/blob/master/PreFreeSurfer/scripts/ACPCAlignment.sh>`_
 * `ANT's DenoiseImage <https://manpages.debian.org/experimental/ants/DenoiseImage.1.en.html>`_
 * `ANT's N4BiasFieldCorrection <http://manpages.ubuntu.com/manpages/trusty/man1/N4BiasFieldCorrection.1.html>`_
 
@@ -21,10 +22,22 @@ Configuring CPAC to run initial preprocessing:
 
 .. figure:: /_images/anat_init_options.png
 
+#. **ACPC Alignment - [On,Off]:** Anterior Commissure - Posterior Comissure (ACPC) alignment. Default is Off. If choose 'on', clicking on the setting icon will bring up a dialog where you can set ACPC alignment parameters.
+
 #. **Non-Local Means Filtering - [On,Off]:** ANTs DenoiseImage. Default is Off.
 
 #. **N4 Bias Field Correction - [On,Off]:** ANTs N4BiasFieldCorrection - a variant of the popular N3 (nonparametric nonuniform normalization) retrospective bias correction algorithm. Default is Off.
 
+Configuring ACPC Alignment options:
+""""""""""""""""""""""""""""""""""""""
+**Note:** These options are pre-set for ACPC Alignment's default values. These do not need to be modified unless you are looking to optimize the results of ACPC alignment for your particular dataset.
+
+.. figure:: /_images/acpc_gui.png
+
+#. **ACPC Brain Size - [150]:** ACPC size of brain in z-dimension in mm. Default: 150mm for human data, 70mm for macaque data.
+#. **ACPC Aligned Skull Template - [path]:** Skull template to be used for ACPC alignment. It is not necessary to change this path unless you intend to use a non-standard template.
+#. **ACPC Aligned Brain Template - [path]:** Brain template to be used for ACPC alignment. For human data, it can be 'None'. It is not necessary to change this path unless you intend to use a non-standard template.
+
 
 Skull-Stripping
 ^^^^^^^^^^^^^^^
@@ -256,7 +269,7 @@ Anatomical Tissue Segmentation
 
 .. raw:: html
 
-    <div class="flowchart-container"><object data="../../_static/flowcharts/segmentation.svg" type="image/svg+xml"></object></div>
+    <div class="flowchart-container"><object data="../_static/flowcharts/segmentation.svg" type="image/svg+xml"></object></div>
 
 C-PAC uses `FSL/FAST <http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FAST>`_ to automatically segment brain images into white matter, gray matter, and CSF. This is done using probability maps that contain information about the likelihood that a given voxel will be of a particular tissue type. Users specify a probability threshold such that voxels meeting a minimum probability of being a particular tissue will be classified as such. This results in masks containing voxels of only a single tissue type.
 
diff --git a/docs/_sources/user/design_a_pipeline.rst b/docs/_sources/user/design_a_pipeline.rst
@@ -28,7 +28,6 @@ Or you can provide it to the C-PAC Singularity container like so:
             -B /tmp:/tmp \
             -B /Users/You/Documents:/configs \
             fcpindi_c-pac_latest-{date}-{hash value}.img s3://fcp-indi/data/Projects/ADHD200/RawDataBIDS /outputs participant --pipeline_file /configs/pipeline_config.yml
-
 **Reporting errors and getting help**
 
 Please report errors on the `C-PAC github page issue tracker <https://github.com/FCP-INDI/C-PAC/issues>`_. Please use the `C-PAC google group <https://groups.google.com/forum/#!forum/cpax_forum>`_ for help using C-PAC and this application.
diff --git a/docs/_sources/user/help.rst b/docs/_sources/user/help.rst
@@ -26,24 +26,28 @@ Crash files are generated by Nipype, and can be viewed from the terminal by typi
 Where :file:`crash-file.pklz` is the name of the crash file you wish to view.
 
 Common Issues
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^
 
-#. My run is crashing with the crash files indicating a memory error.  What is going on?
+#. I have a pipeline configuration that used to work, but now I'm getting errors that start with ``OSError: File /ndmg_atlases/label/Human/``. Why?
 
-	This issue often occurs when scans are resampled to a higher resolution.  This is because functional images, the template that you are resampling to, and the resulting generated image are all loaded into memory at various points in the C-PAC pipeline.  Typically these images are compressed, but they will be uncompressed when loaded.  If the amount of memory required to load the images exceeds the amount of memory available, C-PAC will crash.
+    .. include:: /user/ndmg_atlases.rst
 
-	For instance, suppose you are resampling a 50 MB (100 MB uncompressed) resting-state scan with 3 mm isotropic voxels to a 3 MB (15 MB uncompressed) 1 mm template. By dividing the voxel volume for the original resting state scan by the voxel volume for the template, you can determine that the voxel count of the resampled resting-state scan will be 27 times greater. Therefore, if you multiply the uncompressed file size by 27 you can estimate that the resampled scan alone will take up 2.7 GB of space. You will need at least this much RAM for C-PAC to load the scan.  If you are running multiple subjects simultaneously, you would need to multiply this estimate by the number of subjects.  Note that the template, original image, and any other open applications will also have their own memory requirements, so the estimate would be closer to 2.8 GB per subject plus however much RAM is used before C-PAC is started (assuming no new applications are started mid-run).
+#. My run is crashing with the crash files indicating a memory error.  What is going on?
 
-	To avoid this error, you will need to either get more RAM, run fewer subjects at once, or consider downsampling your template to a lower resolution.
+    This issue often occurs when scans are resampled to a higher resolution.  This is because functional images, the template that you are resampling to, and the resulting generated image are all loaded into memory at various points in the C-PAC pipeline.  Typically these images are compressed, but they will be uncompressed when loaded.  If the amount of memory required to load the images exceeds the amount of memory available, C-PAC will crash.
+    
+    For instance, suppose you are resampling a 50 MB (100 MB uncompressed) resting-state scan with 3 mm isotropic voxels to a 3 MB (15 MB uncompressed) 1 mm template. By dividing the voxel volume for the original resting state scan by the voxel volume for the template, you can determine that the voxel count of the resampled resting-state scan will be 27 times greater. Therefore, if you multiply the uncompressed file size by 27 you can estimate that the resampled scan alone will take up 2.7 GB of space. You will need at least this much RAM for C-PAC to load the scan.  If you are running multiple subjects simultaneously, you would need to multiply this estimate by the number of subjects.  Note that the template, original image, and any other open applications will also have their own memory requirements, so the estimate would be closer to 2.8 GB per subject plus however much RAM is used before C-PAC is started (assuming no new applications are started mid-run).
+    
+    To avoid this error, you will need to either get more RAM, run fewer subjects at once, or consider downsampling your template to a lower resolution.
 
 
     .. _working_dir_crashes:
 
 #. I'm re-running a pipeline, but I am receiving many crashes.  Most of these crashes tell me that a file that has been moved or no longer exists is being used as an input for a step in the C-PAC pipeline.  What is going on and how can I tell C-PAC to use the correct inputs?
 
-	One of the features of Nipype (which C-PAC is built upon) is that steps that have been run before are not re-run when you re-start a pipeline.  Nipype accomplishes this by associating a value with a step based on the properties of that step (i.e., hashing).  Nipype has two potential values that it can associate with a step: a value based on the size and date of the files created by the step, and a value based upon the data present within the files themselves.  The first value is what C-PAC uses with its pipelines, since it is much more computationally practical.  Since this value only looks at the size and date of files to determine whether or not a step has been run, it will not see that the file's path has changed, and it will assume that all paths are consistent with the path structure from when the pipeline was run before.
-
-	To work around this error, you will need to delete the working directory associated with the previous run and create a new directory to replace it for the new run.
+    One of the features of Nipype (which C-PAC is built upon) is that steps that have been run before are not re-run when you re-start a pipeline.  Nipype accomplishes this by associating a value with a step based on the properties of that step (i.e., hashing).  Nipype has two potential values that it can associate with a step: a value based on the size and date of the files created by the step, and a value based upon the data present within the files themselves.  The first value is what C-PAC uses with its pipelines, since it is much more computationally practical.  Since this value only looks at the size and date of files to determine whether or not a step has been run, it will not see that the file's path has changed, and it will assume that all paths are consistent with the path structure from when the pipeline was run before.
+    
+    To work around this error, you will need to delete the working directory associated with the previous run and create a new directory to replace it for the new run.
 
 #. How should I cite C-PAC in my paper?
 
diff --git a/docs/_sources/user/index.rst b/docs/_sources/user/index.rst
@@ -39,6 +39,12 @@ For more information and additional tutorials, check out our `YouTube channel <h
 * `Open science resources for 'Big Data' Analyses of the human connectome <http://www.slideshare.net/CameronCraddock/open-science-resources-for-big-data-analyses-of-the-human-connectome>`_
 * `Computational approaches for mapping the human connectome <http://www.slideshare.net/CameronCraddock/computational-approaches-for-mapping-the-human-connectome>`_
 
+.. _ndmg_atlases:
+
+.. topic:: Neuroparc v1.0: Baseline
+
+  .. include:: /user/ndmg_atlases.rst
+
 .. include:: /user/release_notes/latest.rst
 
 The C-PAC Team
diff --git a/docs/_sources/user/ndmg_atlases.rst b/docs/_sources/user/ndmg_atlases.rst
@@ -0,0 +1,5 @@
+On July 20, 2020, Neuroparc released v1.0. In moving from v0 to v1, the paths to several of the Neuroparc atlases changed. The atlases are used by C-PAC in its default and preconfigured pipelines. C-PAC v1.7.0 includes the Neuroparc v1.0 paths, but **if you are using a pipeline based on C-PAC 1.6.2a or older, you will need to update any Nueroparc v0 paths in your config**
+
+.. csv-table:: All paths begin with ``/ndmg_atlases/label/Human/``
+  :header: "Neuroparc v0","Neuroparc v1"
+  :url: https://raw.githubusercontent.com/FCP-INDI/C-PAC/c89ec698b0dda897fc1e1304810e3c74d8033476/CPAC/resources/templates/ndmg_atlases.csv
diff --git a/docs/_sources/user/pipeline_config.rst b/docs/_sources/user/pipeline_config.rst
@@ -5,6 +5,12 @@ Overview
 
 This section explains how to create a new pipeline or edit an existing one. If you wish to use one of the pre-configured pipelines that come packaged with C-PAC, you can view the current available library of :doc:`pipelines here </user/preconfig>`.
 
+.. _ndmg_atlases:
+
+.. topic:: Neuroparc v1.0: Baseline
+
+  .. include:: /user/ndmg_atlases.rst
+
 There are two ways of setting up or editing a pipeline configuration for C-PAC:
 
 * Using the pipeline configuration interface in the C-PAC GUI
diff --git a/scripts/is_latest.py b/scripts/is_latest.py
@@ -6,8 +6,7 @@
 
 docs_dir = sys.argv[1] if len(sys.argv) > 1 else '/build/docs'
 versions = os.listdir(docs_dir)
-with open('/build/build_version.txt') as bv:
-    build_version = bv.read().strip()
+build_version = os.environ.get('CIRCLE_TAG', '').rstrip('-source')
 if(sort_versions(versions)[1] == build_version):
     sys.exit(0)
 sys.exit(1)
