revert changes to manual

Author: lukaspie

manual/source/classes/contributed_definitions/apm-structure.rst

@@ -40,16 +40,16 @@ The processing steps of ranging and reconstructing are documented as two special
 
 Spatial or other type of filters which are frequently used for atom probe to select specific atom positions
 or portions of the data based on isotopic identity are modeled as base classes for filters, which are
-defined atom-probe-agnostic empower reuse:
+defined atom-probe-agnostic empowering reuse:
 
 :ref:`NXdelocalization`
     Base class to describe the delocalization of point-like objects on a grid.
-    
+
 :ref:`NXisocontour`
     Computational geometry description of isocontouring/phase-fields in Euclidean space.
-    
+
 :ref:`NXmatch_filter`
-    Base class to filter ions based on their type or other descriptors like hit multiplicity.
+    Base class to filter ions based on their type or on other properties like hit multiplicity.
 
 :ref:`NXspatial_filter`
     Base class to filter based on position. This base class takes advantage of :ref:`NXcg_ellipsoid`,
@@ -68,58 +68,47 @@ defined atom-probe-agnostic empower reuse:
 Tools and applications in APM
 #############################
 
-There exist several research software tools in the APM community that deal with handling and analyzing APM data.
-
+Several research software tools exist in the APM community to analyze APM data.
 One of these is the `paraprobe-toolbox <https://paraprobe-toolbox.readthedocs.io/>`_
 The software is developed by `M. Kühbach et al. <https://arxiv.org/abs/2205.13510>`_.
 
 The paraprobe-toolbox is an example of an open-source parallelized software for analyzing
 point cloud data, for assessing meshes in 3D continuum space, and for studying the effects of
-parameterization on descriptors of micro- and nanoscale structural features (crystal defects)
-within materials when characterized and studied with atom probe.
+parameterization on descriptors of micro- and nanoscale structural features (crystal defects).
 
-There is a set of contributed application definitions describing each computational step in the 
-paraprobe-toolbox. These were added to describe the whole workflow in this particular software,
-but can also act as a blueprint for how computational steps of other software tools
-(including commercial ones) could be developed further to benefit from NeXus.
+There is a set of contributed application definitions which describe each computational step
+performed by tools of the paraprobe-toolbox. This is a blueprint of how NeXus can be used
+for documentation also computational steps of other software tools (including commercial ones).
 
 The need for a thorough documentation of the tools was motivated by several needs:
 
 First, users of software would like to better understand and also be able to study for themselves
 which individual parameters and settings for each tool exist and how configuring these
-affects analyses quantitatively. This stresses the aspect how to improve documentation.
+affect analyses quantitatively. This stresses the aspect how to improve documentation.
 
-Second, scientific software like paraprobe-toolbox implement numerical/algorithmical
-(computational) workflows whereby data coming from multiple input sources
-(like previous analysis results) are processed and carried through more involved analyses
-within several steps inside the tool. The tool then creates output as files. This
-provenance and workflow should be documented.
+Second, scientific software like paraprobe-toolbox implements workflows with numerics
+and algorithms that process data from multiple input sources (like previous analysis results),
+and carry these data through multiple steps inside the tool. The tool then creates output as files.
+This provenance and workflow should be documented for reproducibility (the "R" of the FAIR principles
+of data stewardship).
 
-Individual tools of paraprobe-toolbox are developed in C/C++ and/or Python.
-Provenance tracking is useful as it is one component and requirement for making
-workflows exactly numerically reproducible and thus to enable reproducibility
-(the "R" of the FAIR principles of data stewardship).
+Individual tools of the paraprobe-toolbox are developed in C/C++ or Python. Each of these tools
+instructs a workflow that takes three steps each:
+1. The creation of a configuration file.
+2. The actual analysis using a given Python/or C/C++ tool from the toolbox with results summarized in a results file.
+3. The optional analyses/visualization of the results based on data in NeXus/HDF5 files generated by each tool.
 
-For tools of the paraprobe-toolbox each workflow step is a pair or triple of sub-steps:
-1. The creation of a configuration file. 
-2. The actual analysis using a given Python/or C/C++ tool from the toolbox. 
-3. The optional analyses/visualization of the results based on data in NeXus/HDF5 files generated by each tool. 
-
-Data and metadata between the tools are exchanged with NeXus/HDF5 files. This means that data
+Data and metadata between the tools are exchanged via NeXus/HDF5 files. This means that data
 inside HDF5 binary containers are named, formatted, and hierarchically structured according
-to NeXus application definitions.
-
-In a refactoring project, within the FAIRmat project, which is part of the `German
-National Research Data Infrastructure <https://www.nfdi.de/?lang=en>`_, the tools of the
-paraprobe-toolbox were modified to read from and write data using NeXus application definitions.
+to NeXus application definitions. These definitions are specific for each tool:
 
 For example the application definition :ref:`NXapm_paraprobe_surfacer_config`: specifies
-the expectation how a configuration file for the paraprobe-surfacer tool is formatted
-and which parameters it contains including optionality and cardinality constraints.
-
-Thereby, each config file uses a controlled vocabulary of terms. The config files store
-SHA256 checksum for each input file, thereby implementing an uninterrupted
-provenance tracking chain documenting the computational workflow.
+the expected data formatting of a configuration file for the paraprobe-surfacer tool.
+The application definition defines which parameters are expected, which of these
+are optional, and if specific cardinality constraints exist. Each config file defines
+a controlled vocabulary of terms. The config files store SHA256 checksum for each input file,
+thereby implementing an uninterrupted provenance tracking chain that encodes
+the computational workflow.
 
 As an example, a user may first range their reconstruction and then compute spatial
 correlation functions. The config file for the ranging tool stores the files
@@ -131,32 +120,38 @@ imported by the spatial statistics tool which again keeps track of all files
 and reports its results in a spatial statistics tool results file.
 
 This design makes it possible to rigorously trace which numerical results were achieved
-with specific inputs and settings using specifically-versioned tools. Noteworthy,
-this includes Y-junction on a graph which is where multiple input sources are
-combined to generate new results.
+with specific inputs and settings using specifically-versioned tools including 
+Y-junction on the workflow graph where multiple input sources are combined.
+
+Concepts that are used in multiple tools are inherited from the following
+tool-agnostic application definitions:
 
-Defining, documenting, using, and sharing application definitions is a useful and future-proof 
-strategy for software development and data analyses as it enables automated provenance
-tracking working silently in the background.
+:ref:`NXapm_paraprobe_tool_config`, :ref:`NXapm_paraprobe_tool_results`:
+    Configuration and results respectively.
 
-In summary, the following application definitions were defined for the paraprobe-toolbox.
-These are always pairs of application definitions --- one for the configuration (input) side
-and one for the results (output) side. For each tool one such pair is proposed:
+Internally, these inherit from several other tool-agnostic base classes
+adding atom-probe-research-specific concepts:
+
+    :ref:`NXapm_paraprobe_tool_parameters`, :ref:`NXapm_paraprobe_tool_process`, :ref:`NXapm_paraprobe_tool_common`:
+     Parameters, processing specific data, and common parts respectively useful for the application definitions of the tools of the paraprobe-toolbox.
 
 .. _CC-Apm-Paraprobe-Application-Definitions:
 
 Application Definitions
 #######################
 
+In summary, these are the proposed pairs for all tools in the paraprobe-toolbox:
+
 :ref:`NXapm_paraprobe_ranger_config`, :ref:`NXapm_paraprobe_ranger_results`
     Configuration and results respectively of the paraprobe-ranger tool.
     Apply ranging definitions and explore possible molecular ions.
     Store applied ranging definitions and combinatorial analyses of possible iontypes.
 
 :ref:`NXapm_paraprobe_surfacer_config`, :ref:`NXapm_paraprobe_surfacer_results`
     Configuration and results respectively of the paraprobe-surfacer tool.
-    Create a model for the edge of a point cloud via convex hulls, alpha shapes, or alpha-wrappings.
-    Store triangulated surface meshes of models for the edge of a dataset.
+    Create a model for the edge of a point cloud via convex hulls, alpha shapes,
+    or alpha-wrappings. Store triangulated surface meshes of models
+    for the edge of a dataset.
 
 :ref:`NXapm_paraprobe_distancer_config`, :ref:`NXapm_paraprobe_distancer_results`
     Configuration and results respectively of the paraprobe-distancer tool.
@@ -177,17 +172,19 @@ Application Definitions
 
 :ref:`NXapm_paraprobe_nanochem_config`, :ref:`NXapm_paraprobe_nanochem_results`
     Configuration and results respectively of the paraprobe-nanochem tool.
-    Compute delocalization, iso-surfaces, analyze 3D objects, composition profiles, and mesh interfaces.
+    Compute delocalization, iso-surfaces, analyze 3D objects, composition profiles,
+    and mesh interfaces.
 
 :ref:`NXapm_paraprobe_clusterer_config`, :ref:`NXapm_paraprobe_clusterer_results`
     Configuration and results respectively of the paraprobe-clusterer tool.
     Compute cluster analyses with established machine learning algorithms using CPU or GPUs.
 
 :ref:`NXapm_paraprobe_intersector_config`, :ref:`NXapm_paraprobe_intersector_results`
     Configuration and results resepctively of the paraprobe-intersector tool.
-    Analyze volumetric intersections and proximity of 3D objects discretized as triangulated surface meshes
-    in continuum space to study the effect the parameterization of surface extraction algorithms on the
-    resulting shape, spatial arrangement, and colocation of 3D objects via graph-based techniques.
+    Analyze volumetric intersections and proximity of 3D objects discretized as
+    triangulated surface meshes in continuum space to study the effect the
+    parameterization of surface extraction algorithms on the resulting shape,
+    spatial arrangement, and colocation of 3D objects via graph-based techniques.
 
 .. _CC-Apm-Paraprobe-German-NFDI:
 
@@ -197,8 +194,8 @@ Joint work German NFDI consortia NFDI-MatWerk and FAIRmat
 Members of the FAIRmat and the NFDI-MatWerk consortia of the German National Research Data Infrastructure
 are working together within the Infrastructure Use Case IUC09 of the NFDI-MatWerk project to work on examples
 how software tools in both consortia become better documented and interoperable to use. Within this project,
-we have also added the `CompositionSpace tool <https://github.com/eisenforschung/CompositionSpace>`_ by A. Saxena et al. that has been developed at the
-Max Planck Institute for Sustainable Materials in Düsseldorf 
+we added the `CompositionSpace tool <https://github.com/eisenforschung/CompositionSpace>`_ by A. Saxena et al. that has been developed at the
+Max Planck Institute for Sustainable Materials in Düsseldorf using the above-mentioned approach of pairs of application definitions:
 
 :ref:`NXapm_compositionspace_config`, :ref:`NXapm_compositionspace_results`
     Results of a run with Alaukik Saxena's composition space tool.

manual/source/conf.py

@@ -6,8 +6,9 @@
 
 # -- Path setup --------------------------------------------------------------
 
-import sys, os, datetime
-
+import datetime
+import os
+import sys
 
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
@@ -18,10 +19,10 @@
 
 # -- Project information -----------------------------------------------------
 
-project = "nexus"
-author = "NIAC, https://www.nexusformat.org"
-copyright = "1996-{}, {}".format(datetime.datetime.now().year, author)
-description = "NeXus: A Common Data Format for Neutron, X-ray, and Muon Science"
+project = 'NeXus-FAIRmat'
+author = 'The FAIRmat consortium, a member of the German National Research Data Infrastructure (NFDI)'
+copyright = u'2022-{}, {}'.format(datetime.datetime.now().year, author)
+description = u'Proposal of NeXus expansion for FAIRmat data'
 
 # The full version, including alpha/beta/rc tags
 version = "unknown NXDL version"
@@ -35,7 +36,7 @@
 # -- General configuration ---------------------------------------------------
 
 # https://github.com/nexusformat/definitions/issues/659#issuecomment-577438319
-needs_sphinx = "2.3"
+needs_sphinx = '2.5'
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
@@ -52,8 +53,10 @@
     "contrib_ext",
     "chios.bolditalic",
     "sphinx_gallery.gen_gallery",
+    "sphinx_comments",
 ]
 
+
 # Show `.. todo` directives in the output
 # todo_include_todos = True
 
@@ -71,30 +74,80 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-# html_theme = 'alabaster'
-html_theme = "sphinxdoc"
+html_theme = 'alabaster'
+# html_theme = 'sphinxdoc'
+# html_theme = 'sphinx_rtd_theme'
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ["_static"]
 
 # Add extra files
-html_extra_path = ["CNAME"]
-
-html_sidebars = {
-    "**": [
-        "localtoc.html",
-        "relations.html",
-        "sourcelink.html",
-        "searchbox.html",
-        "google_search.html",
-    ],
-}
-
-
-def setup(app):
-    app.add_css_file("details_summary_hide.css")
+html_extra_path = ['CNAME']
+html_logo = 'img/FAIRmat_new.png'
+
+if html_theme== 'sphinx_rtd_theme':
+    html_theme_options = {
+        'logo_only': False,
+        'collapse_navigation': True,
+        'sticky_navigation': True,
+        'navigation_depth': 4,
+        'includehidden': True,
+        'titles_only': False
+    }
+    def setup(app):
+        app.add_css_file('to_rtd.css')
+        app.add_css_file('details_summary_hide.css')
+elif html_theme== 'alabaster': # Alabaster allows a very high degree of control form Sphinx conf.py
+    html_sidebars = {
+    '**': [
+        'about.html',
+        'navigation.html',
+        'localtoc.html',
+        'relations.html',
+        'sourcelink.html',
+        'searchbox.html',
+        'google_search.html'
+        ],
+    }
+    html_theme_options = {
+        'body_text_align': 'justify',
+        'logo_name': True,
+        'github_button': True,
+        'github_type': 'watch',
+        'github_repo': 'nexus_definitions',
+        'github_user': 'FAIRmat-NFDI',
+        'github_count': 'false', # We don't get the cute counter baloon if we want to point to the branch
+        'sidebar_width': '300px',
+        'body_max_width': 'auto',
+        'page_width': 'auto',
+        'font_size': '12pt',
+        'font_family': 'Arial',
+        'description': 'Proposal of NeXus expansion for FAIRmat data.',
+        'show_powered_by': True,
+        'sidebar_header': '#ffffff',
+        'sidebar_hr': '#ffffff',
+        'sidebar_link': '#ffffff',
+        'sidebar_list': '#ffffff',
+        'sidebar_link_underscore': '#ffffff',
+        'sidebar_text': '#ffffff'
+    }
+    def setup(app):
+        app.add_css_file('to_alabaster.css')
+        app.add_css_file('details_summary_hide.css')
+else:
+    html_sidebars = {
+    '**': [
+        'localtoc.html',
+        'relations.html',
+        'sourcelink.html',
+        'searchbox.html',
+        'google_search.html'
+        ],
+    }
+    def setup(app):
+        app.add_css_file('details_summary_hide.css')
 
 
 # The name of an image file (within the static path) to use as favicon of the
@@ -105,6 +158,9 @@ def setup(app):
 # Output file base name for HTML help builder.
 htmlhelp_basename = "NeXusManualdoc"
 
+comments_config = {
+    "hypothesis": True
+}
 html_contexthtml_context = {
     "css_files": [
         "_static/bespoke.css",  # custom CSS styling
@@ -114,12 +170,17 @@ def setup(app):
 
 # -- Options for Latex output -------------------------------------------------
 latex_elements = {
-    "maxlistdepth": 25,  # some application definitions are deeply nested
-    "preamble": r"""
+    'maxlistdepth': 25, # some application definitions are deeply nested
+    'preamble': r'''
     \usepackage{amsbsy}
+    \listfiles
     \DeclareUnicodeCharacter{1F517}{X}
     \DeclareUnicodeCharacter{2906}{<=}
-    \listfiles""",
+    \DeclareUnicodeCharacter{394}{$\Delta$}
+    \DeclareUnicodeCharacter{2206}{$\Delta$}
+    \DeclareUnicodeCharacter{1F517}{X}
+    \DeclareUnicodeCharacter{2906}{<=}
+    \listfiles'''
 }
 
 # -- Options the gallery -------------------------------------------------
@@ -134,3 +195,22 @@ def setup(app):
     "write_computation_times": False,  # disable computation time display
     "show_signature": False,  # disable signature
 }
+
+# -- Inject Git commit information for substitution --------------------------
+
+import subprocess
+
+def run_git_cmd(*args):
+    try:
+        out = subprocess.check_output(('git',) + args)
+        return out.decode().strip()
+    except subprocess.SubprocessError:
+        return 'unknown'
+
+commit_hash = run_git_cmd('rev-parse', '--short', 'HEAD')
+commit_time = run_git_cmd('log', '-1', '--format=%cd', '--date=iso')
+
+rst_epilog = f"""
+.. |commit_hash| replace:: {commit_hash}
+.. |commit_time| replace:: {commit_time}
+"""

manual/source/examples/index.rst

@@ -1,8 +1,8 @@
 .. _Examples:
 
-================================================
-Examples of writing and reading NeXus data files
-================================================
+=============================
+NeXus: Example data files I/O
+=============================
 ..
 	.. image:: ../img/NeXus.png