diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 4fc78b7f4..c49e756fe 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -16,8 +16,8 @@ Documents #[issue number] by @[original issue author] - [ ] My pull request has a descriptive title (not a vague title like `Update index.md`). - [ ] My pull request targets the `source` branch of the repository. - [ ] My commit messages follow best practices. -- [ ] My code follows the [established code style of the repository](../CONTRIBUTING.md). -- [ ] I tried [letting the CI build this branch on a fork or built the project locally](../CONTRIBUTING.md#building) and verified that there are no visible errors. +- [ ] My code follows the [established code style of the repository](../blob/source/CONTRIBUTING.rst). +- [ ] I tried [letting the CI build this branch on a fork or built the project locally](../blob/source/CONTRIBUTING.rst#building) and verified that there are no visible errors. ## Developer Certificate of Origin diff --git a/.gitignore b/.gitignore index a6ee4a168..6ddbbf025 100644 --- a/.gitignore +++ b/.gitignore @@ -1,10 +1,13 @@ *.swp *.py[co] +**/__pycache__ docs/.buildinfo docs/.doctrees docs/objects.inv docs/searchindex.js +docs/_sources/developer/CONTRIBUTING.rst +docs/_sources/references/*.yml docs/_static docs/_images docs/*.html diff --git a/.vscode/launch.json b/.vscode/launch.json deleted file mode 100644 index 8579a2046..000000000 --- a/.vscode/launch.json +++ /dev/null @@ -1,31 +0,0 @@ -{ - "version": "0.2.0", - "configurations": [ - { - "name": "Sphinx Developer", - "type": "python", - "request": "launch", - "cwd": "${workspaceFolder}", - "console": "integratedTerminal", - "pythonPath": "${config:python.pythonPath}", - "debugStdLib": true, - "args": [ - "-m", "sphinx", - "docs/developer/_sources", "docs/developer" - ] - }, - { - "name": "Sphinx User", - "type": "python", - "request": "launch", - "cwd": "${workspaceFolder}", - "console": "integratedTerminal", - "pythonPath": "${config:python.pythonPath}", - "debugStdLib": true, - "args": [ - "-m", "sphinx", - "docs/_sources/user", "docs/user" - ] - } - ] -} \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 837848ce9..000000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,202 +0,0 @@ - - -- [Branches](#branches) - - [Versions](#versions) - - [`nightly`](#nightly) - - [C-PAC release tags](#c-pac-release-tags) -- [Guidelines](#guidelines) - - [Building](#building) - - [Let CircleCI build your drafts / works-in-progress](#let-circleci-build-your-drafts--works-in-progress) - - [Build locally (C-PAC ≥ v1.8.0)](#build-locally-c-pac-≥-v180) -- [Flowcharts](#flowcharts) -- [Tutorials](#tutorials) -- [References and citations](#references-and-citations) -- [Environment notes](#environment-notes) - - - - -## Branches - -Please, always base changes on the `source` branch. `main` branch will be overwritten by the CI deployment. - - -### Versions - - -#### `nightly` - -Pushes to `source` will rebuild docs at https://fcp-indi.github.io/docs/nightly - - -#### C-PAC release tags -Tags in the format `v.*-source` will build docs for the given version. - -If a matching version tag doesn't exist in https://github.com/FCP-INDI/C-PAC/tags, the build should fail. - -Steps to build a release: -1. Checkout a commit from the `source` from a time appropriate for documentation for the branch being built/rebuilt. -2. If you need to make changes, create a branch (`git switch -c` or `git checkout -b`). -3. Make any changes you need for the specific version. -4. Tag the commit that's ready to build (`git tag v`version`-source`) -5. (Force) push to the remote tag (`git push origin v`version`-source`). -6. CircleCI should deploy the versioned documentation. If the version tag is the newest, it should also overwrite `latest` with these documents. - - -## Guidelines - -- Only write a document once, and liberally use the [reStructured Text `.. include::` directive](https://docutils.sourceforge.io/docs/ref/rst/directives.html#include) to include that document where appropriate. -- Use absolute paths for `.. include::`s. That way the path will resolve correctly regardless of differences in nesting levels. -- Include any source documents that you want built in at least one [`toctree`](https://www.sphinx-doc.org/en/1.8/usage/restructuredtext/directives.html#directive-toctree). Use the `:hidden:` option if you don't want it linked in an actual table of contents in the document with the `toctree`. -- Use consistent section title indicators throughout a sourcetree. [fcp-indi.github.io/docs/user](https://fcp-indi.github.io/docs/user) currently has the following hierarchy (top to bottom): - ``` - = - ^ - - - * - # - ` - ' - " - ``` - - -### Building - - -#### Let CircleCI build your drafts / works-in-progress -* Build environment will match actual docs build environment -* CircleCI takes ~2 minutes to build -1. Fork https://github.com/FCP-INDI/fcp-indi.github.io -1. In your fork's settings, set the GitHub Pages `source` to `main` branch - ![GitHub Pages settings example screenshot](./images/github-pages-settings-example.png) -1. Add your project on CircleCI -1. Merge your draft / work-in-progress into your fork's `source` branch. Make sure you push to your fork and not the main repository's `source` branch. -1. Your fork will publish at `https://[your_GitHub_username].github.io/fcp-indi.github.io/`. - - -#### Build locally (C-PAC ≥ v1.8.0) -This documentation aspires to rely on a [single source of truth](https://en.wikipedia.org/wiki/Single_source_of_truth) where possible. To this end, building this documentation requires an installation of the version of [C-PAC](https://github.com/FCP-INDI/C-PAC) that is being documented. - -Steps to build this documentation locally: -1. Clone this repository. -1. _(optional)_
Locally replicate the step "👊 Running cpac commands" from [.circleci/config](./.circleci/config) to generate [cpac](https://pypi.org/project/cpac/) usage strings. - Either perform this "👊 Running cpac commands" step in a separate Python environment or uninstall cpac after generating the usage string(s). - 1. _(optional)_ Create an environment for cpac and activate this environment. - 1. `pip install cpac` - 1. If you don't have a local container for the version of C-PAC you're documenting, `cpac pull` to download the latest or `cpac pull --tag $TAG` to pull a specific version. - 1. Generate ReStructuredText documents with cpac usage strings: - ```BASH - mkdir -p docs/_sources/user/cpac - printf ".. code-block:: console\n\n $ cpac --help\n\n" > docs/_sources/user/cpac/help.rst - cpac --help | sed -e "s/.*/ &/" >> docs/_sources/user/cpac/help.rst - mkdir -p docs/_sources/user/run - printf "Usage: cpac run\n\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\n.. code-block:: console\n\n $ cpac run --help\n\n" > docs/_sources/user/run/help.rst - cpac run --help | sed -e "s/.*/ &/" >> docs/_sources/user/run/help.rst - mkdir -p docs/_sources/user/utils - printf "Usage: cpac utils\n\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\n.. code-block:: console\n\n $ cpac utils --help\n\n" > docs/_sources/user/utils/help.rst - cpac utils --help | sed -e "s/.*/ &/" >> docs/_sources/user/utils/help.rst - ``` - 1. `deactivate` your cpac environment if you used a separate environment or `pip uninstall cpac`. -
-1. Locally install [C-PAC](https://github.com/FCP-INDI/C-PAC) from source. -1. Run `./bin/build $VERSION` where `$VERSION` is the version to build (`nightly`, `latest`, or [semver](https://semver.org/) for production, but this string can be anything you want locally). ![example version](./images/example_version.png) - - -## Flowcharts - -- SVGs exported from Lucidchart have scaling coded in in `width` and `height` XML attributes. Add the XML attributes `preserveAspectRatio="xMinYMin meet"` and `viewBox` to the SVG element in the actual SVG files: - -```xml - -``` - -where `{width}` and `{height}` are the values already present in the existing `width` and `height` XML attributes. -- Load SVGs in HTML `object` elements with the `raw:: html` directive to preserve hyperlinks and scaling: - -```rst -.. raw:: html - - -``` - - -## Tutorials - -See [FCP-INDI/C-PAC_tutorials/CONTRIBUTING.md](https://github.com/FCP-INDI/C-PAC_tutorials/blob/main/CONTRIBUTING.md) for how to contribute tutorials. Add them to the TOC tree in that repository or in any TOC tree in this repository with a relative path beginning `/user/tutorials/` (e.g., `/user/tutorials/observed_usage` for [`observed_usage.ipynb`](https://github.com/FCP-INDI/C-PAC_tutorials/blob/main/observed_usage.ipynb)). - - -## References and citations - -[sphinxcontrib-bibtex](https://sphinxcontrib-bibtex.readthedocs.io/) is installed and configured. This extension creates links between the citations and the reference in the reference list and formats citations in referenced BibTeX files using built-in or [custom styles](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/docs/_sources/references/style.py). To use this Sphinx extension, - -1. Include your references in a BibTeX file. Unless you have a speficic reason to use a separate file, that BibTeX file should be [`docs/_sources/references/references.bib`](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/docs/_sources/references/references.bib). -2. Using the key (the text between the opening `{` and the first `,` in a BibTeX entry) use the ReStructuredText syntax `` :cite:`key` `` to cite your reference in a ReStructuredText file. -3. Include a `.. bibliography::` directive somewhere on any page that you want to use this extension to format references and create two-way links between the references and citations. Both `:cite:` and `.. bibliography::` need to be rendered on the same page for the links to generate. You can list uncited references by listing their BibTeX keys explicilty -4. If you will (or might) use more than one `.. bibliography::` directive on a single rendered page (including `.. include::` directives), choose a prefix for the keys and include that prefix in both the `:cite:` role (like `` :cite:`prefix-key` ``) and the bibliography directive (like `:keyprefix: prefix-`). -5. If you want to include a header over a reference list, use the `.. rubric::` directive above its `.. bibliography` directive. -6. If the entry type (e.g., `book`, `article`, `misc`) of any of the entries in your BibTeX file(s) is not included in [docs/_sources/references/style.py](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/docs/_sources/references/style.py), add a `get_{entry_type}_template` [Pybtex](https://pybtex.org) method to `CPAC_DocsStyle`. - -For example, if you have a BibTeX file called `cpac_citation.bib` that contains - -```BibTeX -@ARTICLE{cpac2013, - AUTHOR={Craddock, Cameron and Sikka, Sharad and Cheung, Brian and Khanuja, Ranjeet and Ghosh, Satrajit S - and Yan, Chaogan and Li, Qingyang and Lurie, Daniel and Vogelstein, Joshua and Burns, Randal and - Colcombe, Stanley and Mennes, Maarten and Kelly, Clare and Di Martino, Adriana and Castellanos, - Francisco Xavier and Milham, Michael}, - TITLE={Towards Automated Analysis of Connectomes: The {Configurable Pipeline for the Analysis of Connectomes (C-PAC)}}, - JOURNAL={Frontiers in Neuroinformatics}, - YEAR={2013}, - NUMBER={42}, - URL={http://www.frontiersin.org/neuroinformatics/10.3389/conf.fninf.2013.09.00042/full}, - DOI={10.3389/conf.fninf.2013.09.00042}, - ISSN={1662-5196} -} -``` - -If you include - -```rst -To cite C-PAC, use :cite:`cite-example-cpac2013`. - -.. rubric Cite C-PAC - -.. bibliography:: - :cited: - :keyprefix: cite-example- -``` - -The rendered file should look something like - -> To cite C-PAC, use [1]. -> -> __Cite C-PAC__ -> -> [1] Craddock, C., Sikka, S., Cheung, B., Khanuja, R., Ghosh, S. S., Yan, C., Li, Q., Lurie, D., Vogelstein, J., Burns, R., Colcombe, S., Mennes, M., Kelly, C., Di Martino, A., Castellanos, F. X., and Milham, M. 2013. [Towards automated analysis of connectomes: the Configurable Pipeline for the Analysis of Connectomes (C-PAC).](http://www.frontiersin.org/neuroinformatics/10.3389/conf.fninf.2013.09.00042/full) *Frontiers in neuroinformatics* 42. doi:[10.3389/conf.fninf.2013.09.00042](https://dx.doi.org/10.3389/conf.fninf.2013.09.00042) - -Or - -```rst -.. bibliography:: - :cited: - - cpac2013 -``` - -to generate the bibliography entry without the citation. - - -## Environment notes -* Because [C-PAC](https://github.com/FCP-INDI/C-PAC.git) and [cpac](https://github.com/FCP-INDI/cpac.git) have conflicting commandline commands, we first run any `cpac` commands in a virtual environment and spoof the `command-output` directive with `code-block` like - ```RST - .. code-block:: console - - cpac run --help - - .. program-output:: cpac_py run --help - :shell: - :ellipsis: 0,9 - ``` -* :heavy_plus_sign: Check [`.circleci/config.yml`](https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/.circleci/config.yml) of the branch you're working from for build dependencies. -* :octocat: Set an environment variable `GITHUBTOKEN` to a [personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line) to increase [your API rate limit](https://developer.github.com/v3/#rate-limiting) from 60 to 5000 requests per hour (for getting [release notes from GitHub](https://github.com/FCP-INDI/C-PAC/releases)). diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst new file mode 100644 index 000000000..3882b7a2a --- /dev/null +++ b/CONTRIBUTING.rst @@ -0,0 +1,303 @@ +Contributing to this documentation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This document is |CONTRIBUTING.rst|_ (in the root of `this repository `_). The other source files (outside of the code itself) are in `this repository`_ under |docs/_sources|_. + +.. contents:: + :local: + +Branches +******** + +Please, always base changes on the ``source`` branch. ``main`` branch will be overwritten by the CI deployment. + +Versions +"""""""" + +``nightly`` +``````````` + +Pushes to ``source`` will rebuild docs at https://fcp-indi.github.io/docs/nightly + +C-PAC release tags +`````````````````` + +Tags in the format ``v.*-source`` will build docs for the given version. + +If a matching version tag doesn't exist in https://github.com/FCP-INDI/C-PAC/tags, the build should fail. + +Steps to build a release: + +#. Checkout a commit from the ``source`` from a time appropriate for documentation for the branch being built/rebuilt. +#. If you need to make changes, create a branch (\ ``git switch -c`` or ``git checkout -b`` ). +#. Make any changes you need for the specific version. +#. Tag the commit that's ready to build (\ ``git tag v`` version\ ``-source`` ) +#. (Force) push to the remote tag (\ ``git push origin v`` version\ ``-source`` ). +#. CircleCI should deploy the versioned documentation. If the version tag is the newest, it should also overwrite ``latest`` with these documents. + +Guidelines +********** + +* Only write a document once, and liberally use the |rst include|_ to include that document where appropriate. +* Use absolute paths for ``.. include::`` s. That way the path will resolve correctly regardless of differences in nesting levels. +* Include any source documents that you want built in at least one |toctree|_. Use the ``:hidden:`` option if you don't want it linked in an actual table of contents in the document with the ``toctree``. +* Use consistent section title indicators throughout a sourcetree. `fcp-indi.github.io/docs/user `_ currently has the following hierarchy (top to bottom): + + .. code-block:: + + = + ^ + - + * + # + ` + ' + " + +Building +"""""""" + +Let CircleCI build your drafts / works-in-progress +`````````````````````````````````````````````````` + +* Build environment will match actual docs build environment +* CircleCI takes ~2 minutes to build + +#. Fork https://github.com/FCP-INDI/fcp-indi.github.io +#. In your fork's settings, set the GitHub Pages ``source`` to ``main`` branch + + .. image:: ./images/github-pages-settings-example.png + :target: ./images/github-pages-settings-example.png + :alt: GitHub Pages settings example screenshot + +#. Add your project on CircleCI +#. Merge your draft / work-in-progress into your fork's ``source`` branch. Make sure you push to your fork and not the main repository's ``source`` branch. +#. Your fork will publish at ``https://[your_GitHub_username].github.io/fcp-indi.github.io/``. + +Build locally (C-PAC ≥ v1.8.0) +`````````````````````````````` + +This documentation aspires to rely on a `single source of truth `_ where possible. To this end, building this documentation requires an installation of the version of `C-PAC `_ that is being documented. + +Steps to build this documentation locally: + +#. Clone `this repository`_. +#. *(optional)* Locally replicate the step "👊 Running cpac commands" from ``.circleci/config`` to generate `cpac `_ usage strings. + Either perform this "👊 Running cpac commands" step in a separate Python environment or uninstall cpac after generating the usage string(s). + + #. *(optional)* Create an environment for cpac and activate this environment. + #. ``pip install cpac`` + #. If you don't have a local container for the version of C-PAC you're documenting, ``cpac pull`` to download the latest or ``cpac pull --tag $TAG`` to pull a specific version. + #. Generate ReStructuredText documents with cpac usage strings: + + .. code-block:: BASH + + mkdir -p docs/_sources/user/cpac + printf ".. code-block:: console\n\n $ cpac --help\n\n" > docs/_sources/user/cpac/help.rst + cpac --help | sed -e "s/.*/ &/" >> docs/_sources/user/cpac/help.rst + mkdir -p docs/_sources/user/run + printf "Usage: cpac run\n\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\n.. code-block:: console\n\n $ cpac run --help\n\n" > docs/_sources/user/run/help.rst + cpac run --help | sed -e "s/.*/ &/" >> docs/_sources/user/run/help.rst + mkdir -p docs/_sources/user/utils + printf "Usage: cpac utils\n\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\`\n.. code-block:: console\n\n $ cpac utils --help\n\n" > docs/_sources/user/utils/help.rst + cpac utils --help | sed -e "s/.*/ &/" >> docs/_sources/user/utils/help.rst + + #. ``deactivate`` your cpac environment if you used a separate environment or ``pip uninstall cpac``. + +#. Locally install `C-PAC`_ from source. +#. Run ``./bin/build $VERSION`` where ``$VERSION`` is the version to build (\ ``nightly`` , ``latest`` , or |semver| for production, but this string can be anything you want locally). + + .. image:: ./images/example_version.png + :target: ./images/example_version.png + :alt: example version + +Flowcharts +********** + +* SVGs exported from Lucidchart have scaling coded in in ``width`` and ``height`` XML attributes. Add the XML attributes ``preserveAspectRatio="xMinYMin meet"`` and ``viewBox`` to the SVG element in the actual SVG files: + +.. code-block:: xml + + + +where ``{width}`` and ``{height}`` are the values already present in the existing ``width`` and ``height`` XML attributes. + + +* Load SVGs in HTML ``object`` elements with the ``raw:: html`` directive to preserve hyperlinks and scaling: + +.. code-block:: rst + + .. raw:: html + + + +Tutorials +********* + +See `FCP-INDI/C-PAC_tutorials/CONTRIBUTING.md `_ for how to contribute tutorials. Add them to the TOC tree in that repository or in any TOC tree in `this repository`_ with a relative path beginning ``/user/tutorials/`` (e.g., ``/user/tutorials/observed_usage`` for |observed_usage|_\ ). + +References and citations +************************ + +`sphinxcontrib-bibtex `_ is installed and configured. This extension creates links between the citations and the reference in the reference list and formats citations in referenced BibTeX files using built-in or `custom styles `_. To use this Sphinx extension, + + +#. Include your references in a BibTeX file. Unless you have a speficic reason to use a separate file, that BibTeX file should be |references.bib|_. +#. If the entry type (e.g., ``book`` , ``article`` , ``misc`` ) of any of the entries in your BibTeX file(s) is not included in `docs/_sources/references/style.py `_\ , add a ``get_{entry_type}_template`` `Pybtex `_ method to ``CPAC_DocsStyle``. +#. If you want to include a header over a reference list, either follow the heirarchy under `the Guidelines section of this document <#guidelines>`_ or use the ``.. rubric::`` directive above its ``.. bibliography`` directive. + +Local bibliography +"""""""""""""""""" + +For a bibliography of just the citations on the current page. This is the bibliography style you probably want, unless you have a specific reason for the central bibliography style. + +#. Using the key (the text between the opening ``{`` and the first ``,`` in a BibTeX entry) use the ReStructuredText syntax |footcite example| to cite your reference in a ReStructuredText file. +#. Somewhere below your citations in the same page, place ``.. footbibliography::``. You can use this directive multiple times. + + .. epigraph:: + + If specified multiple times in the same document, footnotes are only created for references that do not yet have a footnote earlier in the document. + + -- https://sphinxcontrib-bibtex.readthedocs.io/en/2.4.2/usage.html#directive-footbibliography + +Central bibliography +"""""""""""""""""""" + +For a bibliography that includes references from multiple pages, + +#. Include a ``.. bibliography::`` directive somewhere on any page that you want to use this extension to format references and create two-way links between the references and citations. You can list uncited references by listing their BibTeX keys explicilty. +#. If you will (or might) use more than one ``.. bibliography::`` directive (including via ``.. include::`` directives), choose a prefix for the keys and include that prefix in both the ``:cite:`` role (like |prefix-key example| ) and the bibliography directive (like ``:keyprefix: prefix-`` ). + +For example, if you have a BibTeX file called ``cpac_citation.bib`` that contains + +.. code-block:: BibTeX + + @ARTICLE{cpac2013, + AUTHOR={Craddock, Cameron and Sikka, Sharad and Cheung, Brian and Khanuja, Ranjeet and Ghosh, Satrajit S + and Yan, Chaogan and Li, Qingyang and Lurie, Daniel and Vogelstein, Joshua and Burns, Randal and + Colcombe, Stanley and Mennes, Maarten and Kelly, Clare and Di Martino, Adriana and Castellanos, + Francisco Xavier and Milham, Michael}, + TITLE={Towards Automated Analysis of Connectomes: The {Configurable Pipeline for the Analysis of Connectomes (C-PAC)}}, + JOURNAL={Frontiers in Neuroinformatics}, + YEAR={2013}, + NUMBER={42}, + URL={http://www.frontiersin.org/neuroinformatics/10.3389/conf.fninf.2013.09.00042/full}, + DOI={10.3389/conf.fninf.2013.09.00042}, + ISSN={1662-5196} + } + +…and you include + +.. code-block:: rst + + To cite C-PAC, use :cite:`cite-example-cpac2013`. + + .. rubric Cite C-PAC + + .. bibliography:: + :cited: + :keyprefix: cite-example- + +…the rendered file should look something like + +.. highlights:: + + .. raw:: html + +

To cite C-PAC, use [1].

+ +

Cite C-PAC

+ +

[1] Craddock, C., Sikka, S., Cheung, B., Khanuja, R., Ghosh, S. S., Yan, C., Li, Q., Lurie, D., Vogelstein, J., Burns, R., Colcombe, S., Mennes, M., Kelly, C., Di Martino, A., Castellanos, F. X., and Milham, M. 2013. Towards automated analysis of connectomes: the Configurable Pipeline for the Analysis of Connectomes (C-PAC). Frontiers in neuroinformatics 42. doi:10.3389/conf.fninf.2013.09.00042

+ +The local bibliography method to generate the same rendered output: + +.. code-block:: rst + + To cite C-PAC, use :footcite:`cpac2013`. + + .. rubric Cite C-PAC + + .. footbibliography:: + +If you want to generate that bibliography entry without the citation: + +.. code-block:: rst + + .. bibliography:: + :cited: + + cpac2013 + +Environment notes +***************** + +* \:construction_worker: Because `C-PAC`_ and `cpac`_ have conflicting commandline commands, we first run any ``cpac`` commands in a virtual environment and spoof the ``command-output`` directive with ``code-block`` like + + .. code-block:: RST + + .. code-block:: console + + cpac run --help + + .. program-output:: cpac_py run --help + :shell: + :ellipsis: 0,9 + +* \:heavy_plus_sign: Check |circleci config|_ of the branch you're working from for build dependencies. + +* \:octocat: Set an environment variable ``GITHUBTOKEN`` to a `personal access token `_ to increase `your API rate limit `_ from 60 to 5000 requests per hour (for getting `release notes from GitHub `_\ ). + +.. |circleci config| replace:: ``.circleci/config.yml`` + +.. _circleci config: https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/.circleci/config.yml + +.. |CONTRIBUTING.rst| replace:: ``./CONTRIBUTING.rst`` + +.. _CONTRIBUTING.rst: ./CONTRIBUTING.rst + +.. |docs/_sources| replace:: ``./docs/_sources`` + +.. _docs/_sources: ./docs/_sources + +.. |footcite example| raw:: html + + :footcite:`key` + +.. |nightly| replace:: ``nightly`` + +.. _nightly: #nightly + +.. |observed_usage| replace:: ``observed_usage.ipynb`` + +.. _observed_usage: https://github.com/FCP-INDI/C-PAC_tutorials/blob/main/observed_usage.ipynb + +.. |:octocat:| image:: https://github.githubassets.com/images/icons/emoji/octocat.png + :alt: Octocat + :width: 20 + +.. |prefix-key example| raw:: html + + :cite:`prefix-key` + +.. |references.bib| replace:: ``docs/_sources/references/references.bib`` + +.. _references.bib: https://github.com/FCP-INDI/fcp-indi.github.io/blob/source/docs/_sources/references/references.bib + +.. |rst include| replace:: reStructured Text ``.. include::`` directive + +.. _rst include: https://docutils.sourceforge.io/docs/ref/rst/directives.html#include + +.. |semver| raw:: html + + semver + +.. |toctree| replace:: ``toctree`` + +.. _toctree: https://www.sphinx-doc.org/en/1.8/usage/restructuredtext/directives.html#directive-toctree + + +More information +**************** + +For full developer documentation about documentation, see `Documentation `_ \ No newline at end of file diff --git a/README.md b/README.md index d6130c1d4..92a796287 100644 --- a/README.md +++ b/README.md @@ -4,4 +4,4 @@ GitHub Pages repo for FCP-INDI ## Editing -See [CONTRIBUTING](./CONTRIBUTING.md) for editing and contributing guidelines. +See [CONTRIBUTING](./CONTRIBUTING.rst) for editing and contributing guidelines. diff --git a/bin/build b/bin/build index 16bb15f12..a303f1bac 100755 --- a/bin/build +++ b/bin/build @@ -5,6 +5,10 @@ CPAC_DIRECTORY=$(python -c "import os, CPAC; print(os.path.abspath(os.path.dirna ln ${CPAC_DIRECTORY}/resources/configs/1.7-1.8-nesting-mappings.yml docs/_sources/references/1.7-1.8-nesting-mappings.yml || true ln ${CPAC_DIRECTORY}/resources/configs/1.7-1.8-deprecations.yml docs/_sources/references/1.7-1.8-deprecations.yml || true ln ${CPAC_DIRECTORY}/resources/configs/default_pipeline.yml docs/_sources/references/default_pipeline.yml || true +for CONFIG in $(ls ${CPAC_DIRECTORY}/resources/configs/pipeline_config_*.yml) +do + ln ${CONFIG} docs/_sources/references/$(basename ${CONFIG}) || true +done # build docs sphinx-build -b html docs/_sources docs/$1 \ No newline at end of file diff --git a/docs/_sources/conf.py b/docs/_sources/conf.py index 63cdbf3f8..3f701a958 100644 --- a/docs/_sources/conf.py +++ b/docs/_sources/conf.py @@ -18,7 +18,7 @@ import semver import sys -from CPAC import __version__ +from CPAC import _url_version, __version__ from dateutil import parser as dparser from github import Github from github.GithubException import RateLimitExceededException, \ @@ -142,6 +142,8 @@ def yaml_to_rst(path): 'sphinx.ext.autodoc', 'sphinxcontrib.bibtex', 'sphinxcontrib.fulltoc', + 'sphinxemoji.sphinxemoji', + 'sphinx.ext.extlinks', 'sphinx.ext.ifconfig', 'sphinx.ext.intersphinx', 'sphinx.ext.mathjax', @@ -155,6 +157,28 @@ def yaml_to_rst(path): bib.endswith('.bib')] bibtex_default_style = 'cpac_docs_style' +with open(os.path.join(*([os.pardir] * 2), 'CONTRIBUTING.rst'), 'r') as f: + _converted = f.read().replace( + './images', + 'https://github.com/FCP-INDI/fcp-indi.github.io/raw/source/images' + ).replace( + 'fcp-indi.github.io/docs/', + f'fcp-indi.github.io/docs/{_url_version}/' + ).replace('* :octocat:', '* |:octocat:|') + for emoji in ['construction_worker', 'heavy_plus_sign', 'octocat']: + _converted = r'{}'.format(_converted).replace( + r'\:' + f'{emoji}:', f'|:{emoji}:|') + +if _url_version == 'nightly': + _url_version = 'develop' + +extlinks = {'versioned_source': ( + f'https://github.com/FCP-INDI/C-PAC/blob/{_url_version}/%s', + f'{_url_version} source code: ')} + +with open('developer/CONTRIBUTING.rst', 'w') as contributing: + contributing.write(_converted) + # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -191,7 +215,7 @@ def _gh_rate_limit(): MMMMMMMMMMMMMMMMMMM | | MMMMMMMMMMMMMMMMMMM | Set GITHUBTOKEN to your API token in | MMMMMMMMMMMMMMMMMMMMM | your environment to increase rate limit. | - MMMMMMMMMMMMMMMMMMMMMMM | See CONTRIBUTING.md#environment-notes | + MMMMMMMMMMMMMMMMMMMMMMM | See CONTRIBUTING.rst#environment-notes | MMMMMMMMMMMMMMMMMMMMMMMM |_ ______________________________________| MMMM::- -:::::::- -::MMMM |/ MM~:~ 00~:::::~ 00~:~MM @@ -581,6 +605,7 @@ def _unireplace(release_note, unireplace): versions=', '.join(gh_tags[:5]) ) if len(gh_tags) >= 5 else "" + def setup(app): from CPAC.utils.monitoring import custom_logging diff --git a/docs/_sources/developer/documentation.rst b/docs/_sources/developer/documentation.rst new file mode 100644 index 000000000..731816c71 --- /dev/null +++ b/docs/_sources/developer/documentation.rst @@ -0,0 +1,22 @@ +Documentation +============= + +.. contents:: + :local: + +Contributing to this documentation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. include:: /developer/CONTRIBUTING.rst + :start-after: :local: + :end-before: More information + + +Workflows +^^^^^^^^^ + +.. toctree:: + :maxdepth: 2 + :titlesonly: + + workflow_documentation \ No newline at end of file diff --git a/docs/_sources/developer/index.rst b/docs/_sources/developer/index.rst index 362b2be0a..75d152cc3 100644 --- a/docs/_sources/developer/index.rst +++ b/docs/_sources/developer/index.rst @@ -16,11 +16,11 @@ Contents: :titlesonly: installation + documentation pipeline nodes random_state logging - workflow_documentation workflows/index xcpqc testing diff --git a/docs/_sources/developer/xcpqc.rst b/docs/_sources/developer/xcpqc.rst index d71e94eed..7603fa27a 100644 --- a/docs/_sources/developer/xcpqc.rst +++ b/docs/_sources/developer/xcpqc.rst @@ -4,8 +4,7 @@ eXtensible Connectivity Pipeline-style quality control files .. automodule:: CPAC.qc.xcp :members: -.. rubric:: References +References +^^^^^^^^^^ -.. bibliography:: - :cited: - :keyprefix: cite- +.. footbibliography:: diff --git a/docs/_sources/references/references.bib b/docs/_sources/references/references.bib index 49a62dd10..3f379006c 100644 --- a/docs/_sources/references/references.bib +++ b/docs/_sources/references/references.bib @@ -226,6 +226,26 @@ @article{Wang21a pmid = {33789137} } +@incollection{xcp_22, + title = {General Workflow \S{} Processesing Steps}, + author = {{xcp\_d team}}, + year = {2022}, + booktitle = {{xcp\_d} latest version documentation}, + edition = {55eaabc2}, + url = {https://xcp-d.readthedocs.io/en/latest/generalworkflow.html#processesing-steps}, + copyright = {\textcopyright{} Copyright 2021-2022, xcp_d team} +} + +@incollection{xcp_22a, + title = {General Workflow \S{} Preprocessing Steps: {{Confound}} [{{Table}}]}, + booktitle = {{xcp\_d} latest version documentation}, + author = {{xcp\_d team}}, + year = {2022}, + edition = {55eaabc2}, + url = {https://xcp-d.readthedocs.io/en/latest/generalworkflow.html?highlight=36p#id2}, + copyright = {\textcopyright{} Copyright 2021-2022, xcp_d team} +} + @article{Xu19, title = {Interindividual Variability of Functional Connectivity in Awake and Anesthetized Rhesus Macaque Monkeys}, author = {Xu, Ting and Sturgeon, Darrick and Ramirez, Julian S. B. and {Froudist-Walsh}, Se{\'a}n and Margulies, Daniel S. and Schroeder, Charles E. and Fair, Damien A. and Milham, Michael P.}, diff --git a/docs/_sources/references/style.py b/docs/_sources/references/style.py index ad4607c3c..2a8d5148a 100644 --- a/docs/_sources/references/style.py +++ b/docs/_sources/references/style.py @@ -1,5 +1,5 @@ from pybtex.style.formatting import toplevel -from pybtex.style.formatting.plain import Style +from pybtex.style.formatting.unsrt import Style from pybtex.style.names.lastfirst import NameStyle as LastFirst from pybtex.style.sorting.none import SortingStyle as NoSort from pybtex.style.template import field, href, join, optional, \ @@ -14,7 +14,7 @@ class CPAC_DocsStyle(Style): References ---------- - .. [1] Troffaes, M.C.M. 2021. Custom Formatting, Sorting, and Labelling. *sphinxcontrib-bibtex 1.0.0 documentation*. https://sphinxcontrib-bibtex.readthedocs.io/en/1.0.0/usage.html#custom-formatting-sorting-and-labelling + .. [1] Troffaes, M.C.M. 2021. Usage § Custom Formatting, Sorting, and Labelling. *sphinxcontrib-bibtex 2.4.2 documentation*. https://sphinxcontrib-bibtex.readthedocs.io/en/2.4.2/usage.html#bibtex-custom-formatting """ # pylint: disable=line-too-long # noqa: E501 # pylint: disable=no-value-for-parameter,invalid-name def __init__(self, *args, **kwargs): diff --git a/docs/_sources/user/pipelines/preconfig.rst b/docs/_sources/user/pipelines/preconfig.rst index 40b5e5fbd..b51a4735d 100644 --- a/docs/_sources/user/pipelines/preconfig.rst +++ b/docs/_sources/user/pipelines/preconfig.rst @@ -11,7 +11,7 @@ human default: The Default Pipeline ----------------------------- -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/dev/docker_data/default_pipeline.yml `_ +Pipeline Configuration YAML: :versioned_source:`dev/docker_data/default_pipeline.yml` **Note:** C-PAC runs this pipeline by default, and it is not necessary to invoke the `--preconfig` flag to run it. @@ -39,7 +39,7 @@ Several different individual level analysis are performed on the fMRI data inclu anat-only: Default with Anatomical Preprocessing Only ----------------------------------------------------- -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_anat-only.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_anat-only.yml` Based on the preprocessing decisions of the default pipeline, this preconfiguration allows you to immediately kick off a run with only anatomical preprocessing selected. This includes: @@ -50,7 +50,7 @@ Based on the preprocessing decisions of the default pipeline, this preconfigurat preproc: Default without Derivatives ------------------------------------ -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_preproc.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_preproc.yml` Based on the preprocessing decisions of the default pipeline, this preconfiguration allows you to preprocess all of your data, without launching into calculation of outputs and data derivatives. This includes: @@ -71,9 +71,9 @@ Functional: fmriprep-options: fmriprep-Options Pipeline ------------------------------------------- -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_fmriprep-options.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_fmriprep-options.yml` -This pipeline is designed to increase reproducibility with the preprocessing results of the fmriprep pipeline package :cite:`cite-fmriprep-fMRI16` produced by the `Poldrack Lab at Stanford University `_. +This pipeline is designed to increase reproducibility with the preprocessing results of the fmriprep pipeline package :footcite:`fMRI16` produced by the `Poldrack Lab at Stanford University `_. .. bibliography:: :list: bullet @@ -81,15 +81,12 @@ This pipeline is designed to increase reproducibility with the preprocessing res NiPr20 Este19 -.. bibliography:: - :cited: - :keyprefix: cite-fmriprep- - :start: continue +.. footbibliography:: ndmg: Neurodata's 'ndmg-f' Pipeline ----------------------------------- -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_ndmg.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_ndmg.yml` This pipeline is the result of `Neurodata's `_ study to converge upon the intersection of pipeline configuration decisions that maximizes discriminability between participants' data, drawing from the connectome graphs produced (labeled 'ndmg_graph' in the C-PAC output directory). This pipeline invokes a minimal set of preprocessing. @@ -97,7 +94,6 @@ Note, the 'ndmg_graph' connectome graph outputs are always produced by C-PAC. Th .. bibliography:: :list: bullet - :start: continue Kiar18 Neur @@ -106,14 +102,21 @@ Note, the 'ndmg_graph' connectome graph outputs are always produced by C-PAC. Th rbc-options: ReproBrainChart Options Pipeline --------------------------------------------- -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_rbc-options.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_rbc-options.yml` -RBC-options pipeline was built and integrated in C-PAC based on the Reproducible Brain Charts initiative, which aims to aggregate and harmonize phenotypic and neuroimage data to delineate node mechanisms regarding developmental basis of psychopathology in youth and yield reproducible growth charts of brain development :cite:`cite-rbc-Hoff21`. -.. bibliography:: - :cited: - :keyprefix: cite-rbc- - :start: continue +RBC-options pipeline was built and integrated in C-PAC based on the Reproducible Brain Charts initiative, which aims to aggregate and harmonize phenotypic and neuroimage data to delineate node mechanisms regarding developmental basis of psychopathology in youth and yield reproducible growth charts of brain development :footcite:`Hoff21`. + +This pipeline has two regression strategies, 36-parameter (based on XCP-D "36P" :footcite:`xcp_22a`) and acompcor. See the ``Regressors`` section of the pipeline configuration YAML for specific configuration of each of these strategies: + + +.. literalinclude:: /references/pipeline_config_rbc-options.yml + :language: YAML + :start-at: Regressors: + :end-before: # Process and refine masks used to produce regressors and time series for + + +.. footbibliography:: non-human primate ^^^^^^^^^^^^^^^^^ @@ -121,21 +124,17 @@ non-human primate monkey: Default with Monkey Preprocessing ----------------------------------------- -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_monkey.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_monkey.yml` -This pipeline is based on the work of Xu et al. :cite:`cite-monkey-Xu19` and nhp-ABCD-BIDS-pipeline. :cite:`cite-monkey-Stur20` +This pipeline is based on the work of :footcite:t:`Xu19` and nhp-ABCD-BIDS-pipeline :footcite:`Stur20`. .. bibliography:: :list: bullet - :start: continue Wang21a Rami20 -.. bibliography:: - :cited: - :keyprefix: cite-monkey- - :start: continue +.. footbibliography:: Based on the preprocessing decisions of the default pipeline, this preconfiguration allows you to preprocess all of your macaque data, includes: @@ -163,13 +162,13 @@ testing benchmark-ANTS: C-PAC Benchmark with ANTs Registration ------------------------------------------------------ -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_benchmark-ANTS.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_benchmark-ANTS.yml` The benchmark pipeline has remained mostly unchanged since the project's inception, and is used at the end of each release cycle to ensure the results of C-PAC's key outputs have not changed. It is designed to test a wide range of pipeline options. This pipeline is based on registration-to-template using the ANTs/ITK toolset, as this decision impacts many other aspects of the pipeline further downstream. benchmark-FNIRT: C-PAC Benchmark with FSL FNIRT Registration ------------------------------------------------------------ -Pipeline Configuration YAML: `https://github.com/FCP-INDI/C-PAC/blob/main/CPAC/resources/configs/pipeline_config_benchmark-FNIRT.yml `_ +Pipeline Configuration YAML: :versioned_source:`CPAC/resources/configs/pipeline_config_benchmark-FNIRT.yml>` The benchmark pipeline has remained mostly unchanged since the project's inception, and is used at the end of each release cycle to ensure the results of C-PAC's key outputs have not changed. It is designed to test a wide range of pipeline options. This pipeline is based on registration-to-template using the FSL FLIRT & FNIRT, as this decision impacts many other aspects of the pipeline further downstream. diff --git a/docs/_sources/user/tse.rst b/docs/_sources/user/tse.rst index fb1f9dc48..e143f66e3 100644 --- a/docs/_sources/user/tse.rst +++ b/docs/_sources/user/tse.rst @@ -24,7 +24,7 @@ Configuring ROI Time Series Extraction #. **Realignment - [ROI to func, func to ROI]:** Choose functional time-series and ROI realignment method. 'ROI to func' will realign the atlas/ROI to functional space (fast). 'func to ROI' will realign the functional time series to the atlas/ROI space. NOTE: in rare cases, realigning the ROI to the functional space may result in small misalignments for very small ROIs - please double check your data if you see issues. -#. **Connectivity Matrix:** A connectivity matrix can be generated via nilearn :cite:`cite-tse-Abra14,cite-tse-nile21,cite-tse-nile21a`, ndmg :cite:`cite-tse-Kiar18,cite-tse-Neur18` or AFNI for the ``Avg`` timeseries output. +#. **Connectivity Matrix:** A connectivity matrix can be generated via nilearn :footcite:`Abra14,nile21,nile21a`, ndmg :footcite:`Kiar18,Neur18` or AFNI for the ``Avg`` timeseries output. .. include:: /user/pipelines/without_gui.rst @@ -37,7 +37,4 @@ Configuring ROI Time Series Extraction .. rubric:: References -.. bibliography:: - :cited: - :keyprefix: cite-tse- - :start: continue +.. footbibliography:: diff --git a/docs/_sources/user/xcpqc.rst b/docs/_sources/user/xcpqc.rst index 5fc0bfe21..1439c20d1 100644 --- a/docs/_sources/user/xcpqc.rst +++ b/docs/_sources/user/xcpqc.rst @@ -1,8 +1,12 @@ XCP quality control files ========================= -As part of the ReproBrainChart (RBC) project, C-PAC includes quality control summary files (``desc-xcp_quality``) for each ``desc-preproc_bold`` output based on the `eXtensible Connectivity Pipeline (XCP) quality control dictionary `_. +As part of the ReproBrainChart (RBC) project, C-PAC includes quality control summary files (``desc-xcp_quality``) for each ``desc-preproc_bold`` output based on the eXtensible Connectivity Pipeline (XCP) quality control dictionary :footcite:`xcp_22,Ciri19`. .. seealso:: - :doc:`Developer documentation: eXtensible Connectivity Pipeline-style quality control files ` \ No newline at end of file + :doc:`Developer documentation: eXtensible Connectivity Pipeline-style quality control files ` + +References +^^^^^^^^^^ +.. footbibliography:: \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 974c0f190..ba868dabd 100644 --- a/requirements.txt +++ b/requirements.txt @@ -13,5 +13,6 @@ sphinx==4.0 sphinxcontrib-bibtex==2.4.2 sphinxcontrib-fulltoc sphinxcontrib-programoutput +sphinxemoji semver torch