🔀 Merge remote-tracking branch 'origin/source' into excerpts/1.8.1

Author: shnizzedy

.circleci/config.yml

@@ -14,8 +14,9 @@ commands:
           name: ↑ Installing build dependencies
           command: |
             sudo apt-get update && sudo apt-get -y install git python3-dev graphviz graphviz-dev libgraphviz-dev pkg-config python3-sphinx
+            pip install --user -r requirements.txt
             pip install --user -r https://raw.githubusercontent.com/FCP-INDI/C-PAC/<< parameters.version >>/requirements.txt
-            pip install --user git+https://github.com/${CIRCLE_PROJECT_USERNAME}/C-PAC.git@<< parameters.version >> sphinx m2r numpydoc PyGithub "sphinxcontrib-bibtex<2.0" sphinxcontrib-fulltoc sphinxcontrib-programoutput semver torch
+            pip install --user git+https://github.com/${CIRCLE_PROJECT_USERNAME}/C-PAC.git@<< parameters.version >>
             git clone https://github.com/${CIRCLE_PROJECT_USERNAME}/C-PAC.git /home/circleci/build/C-PAC
             cd /home/circleci/build/C-PAC
             git checkout << parameters.version >>
@@ -24,13 +25,6 @@ commands:
             source ~/simple/bin/activate
             pip install cpac
             deactivate
-      - run:
-          name: 🖇️ Linking references
-          command: |
-            CPAC_DIRECTORY=$(python -c "import os, CPAC; print(os.path.abspath(os.path.dirname(CPAC.__file__)))")
-            ln ${CPAC_DIRECTORY}/resources/configs/1.7-1.8-nesting-mappings.yml docs/_sources/references/1.7-1.8-nesting-mappings.yml || true
-            ln ${CPAC_DIRECTORY}/resources/configs/1.7-1.8-deprecations.yml docs/_sources/references/1.7-1.8-deprecations.yml || true
-            curl -o docs/_sources/references/default_pipeline.yml -L https://raw.githubusercontent.com/${CIRCLE_PROJECT_USERNAME}/C-PAC/<< parameters.version >>/dev/docker_data/default_pipeline.yml || true
   
   run-cpac-commands:
     steps:
@@ -39,7 +33,7 @@ commands:
           name: 👊 Running cpac commands
           command: |
             source ~/simple/bin/activate
-            cpac run --help
+            cpac pull
             mkdir -p docs/_sources/user/cpac
             printf ".. code-block:: console\n\n   $ cpac --help\n\n" > docs/_sources/user/cpac/help.rst
             cpac --help | sed -e "s/.*/   &/" >> docs/_sources/user/cpac/help.rst
@@ -58,7 +52,7 @@ commands:
       - run:
           name: 🔧 Configuring git user
           command: |
-            git config --global user.email "[email protected].com"
+            git config --global user.email "[email protected].io"
             git config --global user.name "ci-build"
       - run:
           name: 🔀 Combining new docs with existing docs

CONTRIBUTING.md

@@ -1,13 +1,33 @@
+<!-- TOC -->
+
+- [Branches](#branches)
+    - [Versions](#versions)
+        - [`nightly`](#nightly)
+        - [C-PAC release tags](#c-pac-release-tags)
+- [Guidelines](#guidelines)
+    - [Building](#building)
+        - [Let CircleCI build your drafts / works-in-progress](#let-circleci-build-your-drafts--works-in-progress)
+        - [Build locally (C-PAC ≥ v1.8.0)](#build-locally-c-pac-≥-v180)
+- [Flowcharts](#flowcharts)
+- [References and citations](#references-and-citations)
+- [Environment notes](#environment-notes)
+
+<!-- /TOC -->
+
+<a id="markdown-branches" name="branches"></a>
 ## Branches
 
 Please, always base changes on the `source` branch. `master` branch will be overwritten by the CI deployment.
 
+<a id="markdown-versions" name="versions"></a>
 ### Versions
 
+<a id="markdown-nightly" name="nightly"></a>
 #### `nightly`
 
 Pushes to `source` will rebuild docs at https://fcp-indi.github.io/docs/nightly
 
+<a id="markdown-c-pac-release-tags" name="c-pac-release-tags"></a>
 #### C-PAC release tags
 Tags in the format `v.*-source` will build docs for the given version.
 
@@ -21,12 +41,13 @@ Steps to build a release:
 5. (Force) push to the remote tag (`git push origin v`version`-source`).
 6. CircleCI should deploy the versioned documentation. If the version tag is the newest, it should also overwrite `latest` with these documents. 
 
+<a id="markdown-guidelines" name="guidelines"></a>
 ## Guidelines
 
 - Only write a document once, and liberally use the [reStructured Text `.. include::` directive](https://docutils.sourceforge.io/docs/ref/rst/directives.html#include) to include that document where appropriate.
 - Use absolute paths for `.. include::`s. That way the path will resolve correctly regardless of differences in nesting levels.
 - Include any source documents that you want built in at least one [`toctree`](https://www.sphinx-doc.org/en/1.8/usage/restructuredtext/directives.html#directive-toctree). Use the `:hidden:` option if you don't want it linked in an actual table of contents in the document with the `toctree`.
-- Use consistent section title indicators throughout a sourcetree. [fcp-indi.github.com/docs/user](https://fcp-indi.github.com/docs/user) currently has the following hierarchy (top to bottom):
+- Use consistent section title indicators throughout a sourcetree. [fcp-indi.github.io/docs/user](https://fcp-indi.github.io/docs/user) currently has the following hierarchy (top to bottom):
   ```
   =
   ^
@@ -37,16 +58,50 @@ Steps to build a release:
   '
   "
   ```
-- Let CircleCI build your drafts / works-in-progress
-    * Build environment will match actual docs build environment
-    * CircleCI takes ~2 minutes to build
-    1. Fork https://github.com/FCP-INDI/fcp-indi.github.com
-    1. In your fork's settings, set the GitHub Pages `source` to `master` branch
-        ![GitHub Pages settings example screenshot](./images/github-pages-settings-example.png)
-    1. Add your project on CircleCI
-    1. Merge your draft / work-in-progress into your fork's `source` branch. Make sure you push to your fork and not the main repository's `source` branch.
-    1. Your fork will publish at `https://[your_GitHub_username].github.io/fcp-indi.github.com/`.
 
+<a id="markdown-building" name="building"></a>
+### Building
+
+<a id="markdown-let-circleci-build-your-drafts--works-in-progress" name="let-circleci-build-your-drafts--works-in-progress"></a>
+#### Let CircleCI build your drafts / works-in-progress
+* Build environment will match actual docs build environment
+* CircleCI takes ~2 minutes to build
+1. Fork https://github.com/FCP-INDI/fcp-indi.github.io
+1. In your fork's settings, set the GitHub Pages `source` to `master` branch
+    ![GitHub Pages settings example screenshot](./images/github-pages-settings-example.png)
+1. Add your project on CircleCI
+1. Merge your draft / work-in-progress into your fork's `source` branch. Make sure you push to your fork and not the main repository's `source` branch.
+1. Your fork will publish at `https://[your_GitHub_username].github.io/fcp-indi.github.io/`.
+
+<a id="markdown-build-locally-c-pac-≥-v180" name="build-locally-c-pac-≥-v180"></a>
+#### Build locally (C-PAC ≥ v1.8.0)
+This documentation aspires to rely on a [single source of truth](https://en.wikipedia.org/wiki/Single_source_of_truth) where possible.  To this end, building this documentation requires an installation of the version of [C-PAC](https://github.com/FCP-INDI/C-PAC) that is being documented.
+
+Steps to build this documentation locally:
+1. Clone this repository.
+1. _(optional)_ <details><summary>Locally replicate the step "👊 Running cpac commands" from [.circleci/config](./.circleci/config) to generate [cpac](https://pypi.org/project/cpac/) usage strings.</summary>
+    Either perform this "👊 Running cpac commands" step in a separate Python environment or uninstall cpac after generating the usage string(s).
+    1. _(optional)_ Create an environment for cpac and activate this environment.
+    1. `pip install cpac`
+    1. If you don't have a local container for the version of C-PAC you're documenting, `cpac pull` to download the latest or `cpac pull --tag $TAG` to pull a specific version.
+    1. Generate ReStructuredText documents with cpac usage strings:
+       ```BASH
+        mkdir -p docs/_sources/user/cpac
+        printf ".. code-block:: console\n\n   $ cpac --help\n\n" > docs/_sources/user/cpac/help.rst
+        cpac --help | sed -e "s/.*/   &/" >> docs/_sources/user/cpac/help.rst
+        mkdir -p docs/_sources/user/run
+        printf "Usage: cpac run\n\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\n.. code-block:: console\n\n   $ cpac run --help\n\n" > docs/_sources/user/run/help.rst
+        cpac run --help | sed -e "s/.*/   &/" >> docs/_sources/user/run/help.rst
+        mkdir -p docs/_sources/user/utils
+        printf "Usage: cpac utils\n\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\n.. code-block:: console\n\n   $ cpac utils --help\n\n" > docs/_sources/user/utils/help.rst
+        cpac utils --help | sed -e "s/.*/   &/" >> docs/_sources/user/utils/help.rst
+        ```
+    1. `deactivate` your cpac environment if you used a separate environment or `pip uninstall cpac`.
+    </details>
+1. Locally install [C-PAC](https://github.com/FCP-INDI/C-PAC) from source.
+1. Run `./bin/build $VERSION` where `$VERSION` is the version to build (`nightly`, `latest`, or [<span title='Semantic Versioning'>semver</span>](https://semver.org/) for production, but this string can be anything you want locally). ![example version](./images/example_version.png)
+
+<a id="markdown-flowcharts" name="flowcharts"></a>
 ## Flowcharts
 
 - SVGs exported from Lucidchart have scaling coded in in `width` and `height` XML attributes. Add the XML attributes `preserveAspectRatio="xMinYMin meet"` and `viewBox` to the SVG element in the actual SVG files:
@@ -64,16 +119,17 @@ where `{width}` and `{height}` are the values already present in the existing `w
     <object data="../_static/path/to/chart.svg" type="image/svg+xml"></object>
 ```
 
+<a id="markdown-references-and-citations" name="references-and-citations"></a>
 ## References and citations
 
-[sphinxcontrib-bibtex](https://sphinxcontrib-bibtex.readthedocs.io/) is installed and configured. This extension creates links between the citations and the reference in the reference list and formats citations in referenced BibTeX files using built-in or [custom styles](https://github.com/FCP-INDI/fcp-indi.github.com/blob/source/docs/_sources/references/style.py). To use this Sphinx extension, 
+[sphinxcontrib-bibtex](https://sphinxcontrib-bibtex.readthedocs.io/) is installed and configured. This extension creates links between the citations and the reference in the reference list and formats citations in referenced BibTeX files using built-in or [custom styles](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/docs/_sources/references/style.py). To use this Sphinx extension, 
 
-1. Include your citations in a BibTeX file (see the `*.bib` files in [docs/_sources/references](https://github.com/FCP-INDI/fcp-indi.github.com/blob/source/docs/_sources/references) for examples).
+1. Include your citations in a BibTeX file (see the `*.bib` files in [docs/_sources/references](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/docs/_sources/references) for examples).
 2. Using the key (the text between the opening `{` and the first `,` in a BibTeX entry) use the ReStructuredText syntax `` :cite:`key` `` to cite your reference in a ReStructuredText file.
 3. Include a `.. bibliography::` directive somewhere on any page that you want to use this extension to format references and create two-way links between the references and citations. Specify the (one) BibTeX file for this reference list any formatting for the reference list in this directive. Both `:cite:` and `.. bibliography::` need to be rendered on the same page for the links to generate.
 4. If you will (or might) use more than one `.. bibliography::` directive on a single rendered page (including `.. include::` directives), choose a prefix for the keys and include that prefix in both the `:cite:` role (like `` :cite:`prefix-key` ``) and the bibliography directive (like `:keyprefix: prefix-`).
 5. If you want to include a header over a reference list, use the `.. rubric::` directive above its `.. bibliography` directive.
-6. If the entry type (e.g., `book`, `article`, `misc`) of any of the entries in your BibTeX file(s) is not included in [docs/_sources/references/style.py](https://github.com/FCP-INDI/fcp-indi.github.com/blob/source/docs/_sources/references/style.py), add a `get_{entry_type}_template` [Pybtex](https://pybtex.org) method to `CPAC_DocsStyle`.
+6. If the entry type (e.g., `book`, `article`, `misc`) of any of the entries in your BibTeX file(s) is not included in [docs/_sources/references/style.py](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/docs/_sources/references/style.py), add a `get_{entry_type}_template` [Pybtex](https://pybtex.org) method to `CPAC_DocsStyle`.
 
 For example, if you have a BibTeX file called `cpac_citation.bib` that contains
 
@@ -114,6 +170,7 @@ The rendered file should look something like
 > 
 > <a name="ref1" href="#backref1">[1]</a> Craddock, C., Sikka, S., Cheung, B., Khanuja, R., Ghosh, S. S., Yan, C., Li, Q., Lurie, D., Vogelstein, J., Burns, R., Colcombe, S., Mennes, M., Kelly, C., Di Martino, A., Castellanos, F. X., and Milham, M. 2013. [Towards automated analysis of connectomes: the Configurable Pipeline for the Analysis of Connectomes (C-PAC).](http://www.frontiersin.org/neuroinformatics/10.3389/conf.fninf.2013.09.00042/full) *Frontiers in neuroinformatics* 42. doi:[10.3389/conf.fninf.2013.09.00042](https://dx.doi.org/10.3389/conf.fninf.2013.09.00042)
 
+<a id="markdown-environment-notes" name="environment-notes"></a>
 ## Environment notes
 * Because [C-PAC](https://github.com/FCP-INDI/C-PAC.git) and [cpac](https://github.com/FCP-INDI/cpac.git) have conflicting commandline commands, we first run any `cpac` commands in a virtual environment and spoof the `command-output` directive with `code-block` like 
    ```RST
@@ -125,5 +182,5 @@ The rendered file should look something like
       :shell:
       :ellipsis: 0,9
    ```
-* :heavy_plus_sign: Check [`.circleci/config.yml`](https://github.com/FCP-INDI/fcp-indi.github.com/blob/source/.circleci/config.yml) of the branch you're working from for build dependencies.
+* :heavy_plus_sign: Check [`.circleci/config.yml`](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/.circleci/config.yml) of the branch you're working from for build dependencies.
 * :octocat: Set an environment variable `GITHUBTOKEN` to a [personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line) to increase [your API rate limit](https://developer.github.com/v3/#rate-limiting) from 60 to 5000 requests per hour (for getting [release notes from GitHub](https://github.com/FCP-INDI/C-PAC/releases)).

GUI/index.html

@@ -1,5 +1,5 @@
 <!DOCTYPE html>
 <meta charset="utf-8">
 <script>
-window.location.href = "https://fcp-indi.github.com/C-PAC_GUI" + window.location.href.split('/GUI')[1]
+window.location.href = "https://fcp-indi.github.io/C-PAC_GUI" + window.location.href.split('/GUI')[1]
 </script>

README.md

@@ -1,4 +1,4 @@
-# fcp-indi.github.com
+# fcp-indi.github.io
 
 GitHub Pages repo for FCP-INDI
 

bin/build

@@ -1,3 +1,10 @@
 #!/bin/bash
 
+# link references
+CPAC_DIRECTORY=$(python -c "import os, CPAC; print(os.path.abspath(os.path.dirname(CPAC.__file__)))")
+ln ${CPAC_DIRECTORY}/resources/configs/1.7-1.8-nesting-mappings.yml docs/_sources/references/1.7-1.8-nesting-mappings.yml || true
+ln ${CPAC_DIRECTORY}/resources/configs/1.7-1.8-deprecations.yml docs/_sources/references/1.7-1.8-deprecations.yml || true
+ln ${CPAC_DIRECTORY}/resources/configs/default_pipeline.yml docs/_sources/references/default_pipeline.yml || true
+
+# build docs
 sphinx-build -b html docs/_sources docs/$1

docs/_sources/_images/anat_surface.png

@@ -27,7 +27,7 @@
 
 sys.path.append(os.path.dirname(__file__))
 
-from references import CPAC_DocsStyle
+from references import CPAC_DocsStyle  # noqa: E402
 
 register_plugin('pybtex.style.formatting', 'cpac_docs_style', CPAC_DocsStyle)
 

docs/_sources/conf.py

@@ -9,7 +9,7 @@ ALFF is defined as the total power within the frequency range between 0.01 and 0
 
 Computation and Analysis Considerations
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-All computations are performed in a subject's native space. After transforming voxel time series frequency information into the power domain, calculation of these measures is relatively simple. ALFF is calculated as the sum of amplitudes within a specific low frequency range. f/ALFF is calculated as a fraction of the sum of amplitudes across the entire frequency range detectable in a given signal. For both measures, amplitudes in subject-level maps are transformed into Z-scores to create standardized subject-level maps. Anatomical images and Z-score maps are then transformed into MNI152 standard space. For more detail on how CPAC computes these steps, please see the `ALFF and f/ALFF Page of the developer documentation <http://fcp-indi.github.com/docs/developer/workflows/alff.html>`_.
+All computations are performed in a subject's native space. After transforming voxel time series frequency information into the power domain, calculation of these measures is relatively simple. ALFF is calculated as the sum of amplitudes within a specific low frequency range. f/ALFF is calculated as a fraction of the sum of amplitudes across the entire frequency range detectable in a given signal. For both measures, amplitudes in subject-level maps are transformed into Z-scores to create standardized subject-level maps. Anatomical images and Z-score maps are then transformed into MNI152 standard space. For more detail on how CPAC computes these steps, please see the `ALFF and f/ALFF Page of the developer documentation <http://fcp-indi.github.io/docs/developer/workflows/alff.html>`_.
 
 Though both ALFF and f/ALFF are sensitive mostly to signal from gray matter, ALFF is more prone to noise from physiological sources, particularly near the ventricles and large blood vessels (Zuo et al., 2008;2010). The figure below (from Zuo et al., 2010) shows areas in which ALFF shows higher amplitude than f/ALFF, as well as the relative sensitivity of these measures to gray matter.
 

docs/_sources/user/alff.rst

@@ -5,6 +5,27 @@
 
     <div class="flowchart-container"><object data="../_static/flowcharts/anatomical.svg" type="image/svg+xml"></object></div>
 
+Surface Analysis
+^^^^^^^^^^^^^^^^
+
+Surface analysis runs FreeSurfer recon-all and generates CSF, WM, GM masks, pial surface mesh, smoothed surface mesh, spherical surface mesh, white matter surface mesh, sulcal depth surface maps, cortical thickness surface maps and cortical volume surface maps.
+
+Configuring CPAC to run surface analysis:
+"""""""""""""""""""""""""""""""""""""""""
+
+.. figure:: /_images/anat_surface.png
+
+#. **FreeSurfer - [On,Off]:** FreeSurfer recon-all. Default is Off.
+
+Configuration Without the GUI
+""""""""""""""""""""""""""""""
+
+The following nested key/value pairs that will be set to these defaults if not defined in your :doc:`pipeline configuration YAML </user/pipelines/pipeline_config>`:
+
+.. literalinclude:: /references/default_pipeline.yml
+   :language: YAML
+   :lines: 152-159
+
 Initial Preprocessing
 ^^^^^^^^^^^^^^^^^^^^^