more work on spec and dictionary

Author: bruceravel

README.md

@@ -14,16 +14,12 @@ files containing single-scan XAS data.
 not yet been released.**
 
 
-Specification Document
-------------------------
+Specification Documents
+-----------------------
 
+ * The [XDI specification](specification/spec.md), version 1.0.
 
-The [XDI specification](specification/spec.md), version 1.0.
-
-
-The
-[metadata dictionary](https://github.com/XraySpectroscopy/XAS-Data-Interchange/wiki/Dictionary-of-metadata)
-is also a GitHub wiki
+ * The [metadata dictionary](specification/dictionary.md), version 1.0.
 
 
 
@@ -66,9 +62,11 @@ Other files
    validate as XDI
  * The `doc/` folder contains the LaTeX source of the poster on XDI
    presented at the XAFS15 conference
- * The `magic` file can be appended to `/etc/magic` or otherwise used
-   by the Unix file determination system:
+ * The `filemagic` folder contains tools for use by the Unix file
+   determination system:
 
         ~> file -m magic data/co_metal_rt.xdi 
 		data/co_metal_rt.xdi: XAS Data Interchange file -- XDI specification 1.0
 
+   It also defines an icon and a registry entry for XDI files on
+   Windows.

filemagic/README.md

@@ -0,0 +1,33 @@
+Tools for using XDI files
+=========================
+
+# Unix computers
+
+ * `magic`: file type difinition for XDI for the [Unix file type recognition system](http://en.wikipedia.org/wiki/File_%28command%29)
+ * `install_magic`: Bash shell script for installing the contents of the `magic` file
+
+
+Install XDI file magic as root:
+
+        ~> sudo ./install_magic
+
+This assumes that the file `/etc/magic` is used for local magic data.
+If the local file magic file is in some other location on your
+computer, then
+
+        ~> sudo ./install_magic /path/to/magic
+
+where `/path/to/magic` is that location on your computer.
+
+Once installed, XDI files can be identified with the [file command](http://en.wikipedia.org/wiki/File_%28command%29):
+
+	    ~> file -m magic data/co_metal_rt.xdi 
+		data/co_metal_rt.xdi: XAS Data Interchange file -- XDI specification 1.0
+
+# Windows computers
+
+ * `xdi.ico`: Windows icon for XDI files
+ * `xdi.reg`: Windows registry entry for XDI files
+
+Double click on the `xdi.reg` file to modify a computer's registry.
+Once this is done, XDI files will be displayed using the XDI icon.

specification/README.md

@@ -3,20 +3,24 @@ XDI Specification Documents
 
  * [XDI Specification](spec.md): The XAS Data Interchange specification [(PDF file)](xdi_spec.pdf)
 
+ * [Dictionary of Metadata](dictionary.md): The XDI dictionary [(PDF file)](xdi_dictionary.pdf)
+
  * [XDI background](background.md): Background on what XDI can be used for [(PDF file)](xdi_background.pdf)
 
- * [Dictionary of Metadata](dictionary.md): The XDI dictionary [(PDF file)](xdi_dictionary.pdf)
+## Generating PDF versions of these documents
 
-## Generating PDF version of the specification documents
+The following software is used to convert the markdown source files
+into nice-lookingt PDF files:
 
-The file [maketex.sh](maketex.sh) is a Bash shell script which uses
-[Pandoc](http://johnmacfarlane.net/pandoc/) and
-[sed](http://www.gnu.org/software/sed/) to convert the markdown source
-into latex and to apply several hand-crafted customizations to make
-the resulting PDF documents look lovely.
+1. [Pandoc](http://johnmacfarlane.net/pandoc/): convert markdown into LaTeX.
+1. [sed](http://www.gnu.org/software/sed/): apply several customizations to the LaTeX files
+1. [pdflatex](https://www.tug.org/texlive/): compile the LaTeX files into PDF
 
-The latex files are then compiled into PDF using
-[pdflatex](https://www.tug.org/texlive/).
+The file [maketex.sh](maketex.sh) is a Bash shell script which
+automates the first two steps.
+
+The file [xdi.sty](xdi.sty) is a LeTeX style file which controls the
+appearance of the PDF output
 
 **To make the XDI Specification document:**
 
@@ -29,14 +33,6 @@ pdflatex twice to get all the internal references correct.
 
 This makes a PDF file called `xdi_spec.pdf`.
 
-**To make the XDI Background document:**
-
-        ~> ./maketex.sh background
-        ~> pdflatex xdi_background.tex
-        ~> pdflatex xdi_background.tex
-
-This makes a PDF file called `xdi_background.pdf`.
-
 **To make the XDI Dictionary of Metadata document:**
 
         ~> ./maketex.sh dictionary
@@ -45,3 +41,11 @@ This makes a PDF file called `xdi_background.pdf`.
 
 This makes a PDF file called `xdi_dictionary.pdf`.
 
+**To make the XDI Background document:**
+
+        ~> ./maketex.sh background
+        ~> pdflatex xdi_background.tex
+        ~> pdflatex xdi_background.tex
+
+This makes a PDF file called `xdi_background.pdf`.
+

specification/background.md

@@ -4,14 +4,16 @@ Use cases for the XAS Data Interchange format
 # Conventional XAS
 
 In a conventional XAS experiment, we measure a sample somewhere
-between 2 and 10,000 scans, possibly requiring dead-time or other
+between 2 and 10,000 times, possibly requiring dead-time or other
 corrections.  Some data processing is required to correct, calibrate,
 and/or align the data.  Those scans are then merged into a single
 spectrum.
 
 ![The merge of several dozen scans at the Cd K edge on a sample dilute in Cd.](images/convxas.png)
 
-XDI is about how we express the merged spectrum.
+XDI is about how we express the merged spectrum.  In some case, XDI
+may also be a suitable format for the individual XAS measurements, as
+well.
 
 # XRF imaging experiments
 
@@ -30,16 +32,16 @@ the imaging experiment.
 # Difraction anomalous fine structure (DAFS)
 
 An anomalous scattering experiments yields energy-dependent scattering
-intensities.  Here we see (\footnote{Ravel et al. PRB 60, 778-785
-(1999)
-[doi:10.1103/PhysRevB.60.778](http://dx.doi.org/10.1103/PhysRevB.60.778)
-DAFS data measured near the Ti K and Ba L3 edges on BaTiO3.  From
-these data, mu(E) or chi(k) spectra are extracted and interpreted as
-position-selective EXAFS.
+intensities.  Here we see DAFS data measured (Ravel et al. PRB 60,
+778-785 (1999)
+[doi:10.1103/PhysRevB.60.778](http://dx.doi.org/10.1103/PhysRevB.60.778))
+near the Ti K and Ba L3 edges on BaTiO3.  From these data, mu(E) or
+chi(k) spectra are extracted and interpreted as position-selective
+EXAFS.
 
 ![A DAFS measurement on BaTiO3 and the chi(k) spectrum extracted from it.](images/dafs.png)
 
-XDI is about how we express the mu(E) or chi(k) spectra extracted from
+XDI is about how we express the mu(E) or chi(k) spectrum extracted from
 the anomalous diffraction measurement.
 
 
@@ -53,5 +55,5 @@ scattering. (Bergmann, et al. Chem. Phys. Lett. 369 184 (2003)
 
 ![NIXS data measured on graphite and the XANES spectrum extracted from it.](images/nixs.png)
 
-XDI is about how we express the mu(E) spectra extracted from the
+XDI is about how we express the mu(E) spectrum extracted from the
 non-resonant inelastic scattering measurement.

specification/dictionary.md

@@ -1,5 +1,5 @@
-Dictionary of XDI Metadata
-==========================
+Dictionary of XAS Data Interchange Metadata
+===========================================
 
 XDI Working Group
 -----------------
@@ -21,7 +21,7 @@ includes:
 1. The name representing the datum
 1. The meaning of the datum
 1. Thw units of the datum
-1. The format of representing its value
+1. The format for representing its value
 
 Words used to signify the requirements in the specification **shall**
 follow the practice of
@@ -49,13 +49,13 @@ multi-spectral datasets.
 ## The XDI syntax
 
 This dictionary has been developed along with the
-[XDI specification](spec.md).  Any examples given in this dictionary
-use the **recommended** XDI syntax.  The metadata name consists of the
-capitalized namespace, followed by a dot, followed by a tag.  Here
-is an example: `Element.symbol`.  When appearing in an XDI file to
-convey a metadata value, the line begins with a comment token and end
-with an end-of-line token.  A colon is the delimiting token between
-the metadata name and its value.  Here is an example:
+[XDI specification](spec.md).  All examples given in this dictionary
+use all recommendations of the XDI syntax.  The metadata name consists
+of the capitalized namespace, followed by a dot, followed by a tag.
+Here is an example: `Element.symbol`.  When appearing in an XDI file
+to convey a metadata value, the line begins with a comment token and
+end with an end-of-line token.  A colon is the delimiting token
+between the metadata name and its value.  Here is an example:
 
        # Element.symbol: Cu
 
@@ -74,7 +74,7 @@ their definitions.
 
 * _free-format string_: This is a string which can contain any
   character (save end-of-line characters) in any encoding system.  A
-  free-sormat string need not be ASCII and need not be English.
+  free-format string need not be ASCII and need not be English.
   Because applications using XDI may not be capable of handling some
   encoding systems, it is **recommended** that free-format strings be
   ASCII.
@@ -83,31 +83,34 @@ their definitions.
   white space, followed by a string denoting the units of the previous
   string.  As an example, a value for `Column.1` might be `energy eV`,
   which identifies the contents of the first column in the data table
-  as containing energy values expressed in electron volt units.
+  as containing energy values expressed in electron volt units.  The
+  selection of possible units for a tag is given in the definition of
+  the tag.
 
 * _float_: A float is a string which is interpretable as a
   floating-point number in the C programming language.  An integer is
   permissable.  Values of `NaN`, `sNAN`, `qNAN`, `inf`, `+inf`, and
-  `-inf` are not allowed in XDI.  That is, a float in XDI must be
-  finite number.  See [IEEE 754-2008](http://grouper.ieee.org/groups/754/).
+  `-inf` are not allowed in XDI.  That is, a float in XDI **must** be
+  a finite number.  See
+  [IEEE 754-2008](http://grouper.ieee.org/groups/754/).
 
 * _float + units_: This is a float as defined above, followed by white
   space, followed by a string identifying the units of the number.
   For example, a value for `Sample.temperature`, which identifies the
-  tamperature at which an XAS measurement is made, might be `500 K`,
-  which identifies the temperature of the measurements in Kelvin
+  temperature at which an XAS measurement is made, might be `500 K`,
+  identifying the temperature of the measurements in Kelvin
   temperature units.  The selection of possible units for a tag is
   given in the definition of the tag.
 
 * _chemical formulas_: `Sample.stoichiometry` is intended to represent
-  the elemental composition of the sample.  To make these formulas
-  interpretable by computer, this and extension fields which represent
-  chemical information **must** use the
-  [IUCr definition of a chemical formula]( http://www.iucr.org/__data/iucr/cifdic_html/1/cif_core.dic/Cchemical_formula.html).
+  the elemental composition of the sample.  To allow interpretation of
+  chemical formulas by computer, this field and extension fields which
+  represent chemical information **must** use the
+  [IUCr definition of a chemical formula](http://www.iucr.org/__data/iucr/cifdic_html/1/cif_core.dic/Cchemical_formula.html).
 
 * _time_: Because of the wide variability of cultural standards in the
   representation of time, XDI defines a strict standard for time
-  stamps in XDI files.  `Scan.start_time`, `Scan_end_time`, and any
+  stamps in XDI files.  `Scan.start_time`, `Scan.end_time`, and any
   extension fields dealing in time **must** use the
   [ISO 8601 specification for combined dates and times](http://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations)
 
@@ -126,6 +129,10 @@ their definitions.
 
 Some additional comments:
 
+* Locale is **not** respected when interpreting floating point numbers.
+  The decimal mark **must** be a dot (`.`, ASCII 46).  The decimal mark
+  **must not** be a comma (`,`, ASCII 44).
+
 * A tag which is in a defined family but which is not defined in this
   dictionary **must** be interpreted as have a free-format string as its
   value.
@@ -352,7 +359,7 @@ the signal chain behind the detector.
 * **Namespace:** `Sample` -- **Tag:** `stoichiometry`
      * _Description_: The stoichiometric formula of the measured sample 
      * _Units_: none
-     * _Format_: see [the CIF definition of chemical_formula](http://www.iucr.org/__data/iucr/cifdic_html/1/cif_core.dic/Cchemical_formula.html)
+     * _Format_: see [the IUCr definition of chemical_formula](http://www.iucr.org/__data/iucr/cifdic_html/1/cif_core.dic/Cchemical_formula.html)
 
 * **Namespace:** `Sample` -- **Tag:** `prep`
      * _Description_: A string summarizing the method of sample preparation
@@ -394,7 +401,7 @@ as defined fields in future versions of the XDI specification.
 * `Sample.opacity`
 * `Sample.electrochemical_potential`
 
-Almost all of these examples **should** take a float+units as its value.
+Almost all of these examples **should** take a float+units as values.
 
 
 
@@ -586,4 +593,4 @@ data acquisition, data analysis, and data archiving software.
 
 If an extension tag is not understood due its lack of defined
 semantics, the **recommended** behavior for software touching
-the data be to silently preserve the metadata.
+the data is to silently preserve the metadata.

specification/maketex.sh

@@ -22,16 +22,15 @@ elif [ "$file" = "background" ]; then
     sed -i '/section.Conve/i \\\linenumbers\\linenumbersep=25pt\n' temp1.tex
     sed -i 's/includegraphics/includegraphics[height=100px]/' temp1.tex
 elif [ "$file" = "dictionary" ]; then
-    sed -i 's/section{Dictionary of/section*{Dictionary of Data/' temp1.tex
+    sed -i 's/section{Dictionary of/section*{Dictionary of/' temp1.tex
     sed -i 's/subsection{XDI Working Group/\subsection*{XDI Working Group/' temp1.tex
     sed -i '/section.Overview/i \\\tableofcontents\\thispagestyle{empty}\\newpage\\linenumbers\\linenumbersep=25pt\n\n~\n' temp1.tex
 else
     echo "$file not valid"
     exit
 fi
 
-##sed  's/\\hyperref[(.*)]{.*}/Sec.~\\ref{$1}/' temp1.tex
-sed -i 's/\\hyperref\[\(.*\)\]{.*}/Sec.~\\ref{\1}/' temp1.tex
+sed -i 's/\\hyperref\[\([^]]*\)\]{[^}]*}/Sec.~\\ref{\1}/g' temp1.tex
 
 echo -n "xdi and xditt commands ..."
 ## use the \xdi macro