merge from master

Author: rscohn2

README.rst

@@ -28,7 +28,7 @@ The simplest and quickest way to edit is directly in the GitHub web
 interface. It has an editor, previewer, and lets you commit
 changes. You won't need to install any local tools. The previewer
 knows how to render RST, but not the sphinx directives so it will not
-display exactly as the real manual.
+display exactly as the real document.
 
 -------------------------
 Working with a Local Copy
@@ -42,7 +42,7 @@ to do the same task manually.
 Setup
 -----
 
-Install Python 3 and doxygen (>= 1.8.17).  To install on **Ubuntu**::
+Install Python 3, doxygen (>= 1.8.17), latex, etc.  To install on **Ubuntu**::
 
    sudo scripts/install.sh
 
@@ -60,9 +60,6 @@ On Windows::
   python scripts\oneapi.py spec-venv
   spec-venv\Scripts\activate
   
-MKL and Level Zero are temporarily in separate private repos. If you have access to the repos, you can clone them::
-
-  python scripts/oneapi.py clones
 
 Building the docs
 -----------------
@@ -71,10 +68,6 @@ To build the html document::
 
   python scripts/oneapi.py html
 
-This will not work on Windows because we are using symbolic links for
-the elements that are in separate repos. However, Windows can build
-individual specs for individual elements.
-
 The document is organized as a book with chapters. Each element of
 oneAPI is its own chapter and can be built separately. For example, to
 build the oneVPL chapter, do::
@@ -116,6 +109,18 @@ size of the text. Probably a lot more.
 linting. I could not find any support for rejustifying paragraphs to
 80 characters, which makes it difficult to use.
 
+------------------
+Submitting changes
+------------------
+
+Changes are submitted as PR's to this repo. It's up to you how you get 
+to the point of making the PR. If you have write access to this repo, you
+can push a feature branch to this repo, and then do the PR from the feature
+branch. If you work this way, the CI will publish HTML, PDF to the staging
+server. You can also fork this repo and do the PR from your fork. CI will
+build the document and save the results as an artifact, but will not
+publish to staging server.
+
 ------
 Docker
 ------
@@ -137,11 +142,11 @@ CI
 
 We use GitHub actions. See `<.github/workflows/main.yml>`_
 
-On every commit, the CI system builds and publishes the document to
+On every commit to every branch, the CI system builds and publishes the document to
 the staging server. To see the URL, look at the end of the log for the
 build step in the CI system. The staging server is an s3 bucket, and
 the access keys are managed as GitHub action secrets. PR's based on
-forks do not have access to the keys and will not publish on the
+forks do not have access to the keys and will build, but not publish on the
 staging server.
 
 For commits to the publish branch, the document is staged inside a
@@ -164,9 +169,7 @@ production with::
 
   python scripts/oneapi.py prod-publish
 
-Then purge the CDN. Generate a list of URLs with::
-
-  python scripts/oneapi.py purge
+Then purge the CDN. 
 
 ------------
 More Reading

roadmap.rst

@@ -254,6 +254,13 @@ Summary
 - oneVPL
 
   - Breaking changes introduced.
+  
+=========  ==========
+Date       Milestone
+=========  ==========
+6/25/2020  All components have merged to master
+6/26/2020  Release
+=========  ==========
 
 0.9.0
 -----

source/_static/custom.css

@@ -16,6 +16,16 @@
     margin-right: -0.7em;
 }
 
+/* A workaround for https://github.com/readthedocs/sphinx_rtd_theme/issues/647
+ * Override display for function signatures so that there is spacing between
+ * types and arguments */
+.rst-content dl:not(.docutils) dt {
+    display: table-cell !important;
+}
+.rst-content dl:not(.docutils) dd {
+    margin-top: 6px;
+}
+
 /* override table width restrictions
  * https://rackerlabs.github.io/docs-rackspace/tools/rtd-tables.html */
 @media screen and (min-width: 767px) {

source/conf.py

@@ -88,7 +88,8 @@
 # documentation.
 #
 html_theme_options = {
-  'includehidden': False
+  'includehidden': False,
+  'collapse_navigation': False
 }
 
 html_context = {

source/conf/common_conf.py

@@ -2,6 +2,8 @@
 import sys
 import string
 
+import docutils
+
 def path_relative_to_repo_root(relative_path):
     this_dir = os.path.dirname(os.path.realpath(__file__))
     root_dir = os.path.abspath(os.path.join(this_dir, '../..'))
@@ -57,6 +59,10 @@ def path_relative_to_repo_root(relative_path):
 .. |mkl_full_name| replace:: oneAPI Math Kernel Library
 .. |mkl_version| replace:: $oneapi_version
 .. _`Level Zero Specification`: https://spec.oneapi.com/level-zero/latest/index.html
+.. include:: <isonum.txt>
+.. |regsup| replace:: :supsub:`reg`
+.. |intel_r| replace:: Intel\ :supsub:`reg`
+.. |msdk_full_name| replace:: Intel\ :supsub:`reg` Media Software Development Kit
 """)
 
 rst_prolog = prolog_template.substitute(env)
@@ -90,6 +96,13 @@ def path_relative_to_repo_root(relative_path):
     'extraclassoptions': 'openany,oneside'
 }
 
+def supsub_role(name, rawtext, text, lineno, inliner, options={}, content=[]):
+    node = docutils.nodes.superscript()
+    node2 = docutils.nodes.substitution_reference(refname=text)
+    node += [node2]
+    return [node],[]
+
 def setup(app):
+    app.add_role('supsub', supsub_role)
     add_custom_css = getattr(app,'add_css_file',getattr(app,'add_stylesheet'))
     add_custom_css('custom.css')

source/elements/oneDAL/Makefile

@@ -1,6 +1,6 @@
 GH_PAGES = build/gh-pages
 
-.PHONY: build doxygen parse-doxygen clean gh-pages
+.PHONY: html pdf doxygen parse-doxygen clean gh-pages
 
 html:
 	sphinx-build -M html source build -q
@@ -16,9 +16,11 @@ parse-doxygen: doxygen
 	python -m dalapi.doxypy.cli doxygen/xml --compact > build/tree.yaml
 
 clean:
+	rm -rf build/doctrees
 	rm -rf build/html
+	rm -rf build/latex
 
-gh-pages: build
+gh-pages: html
 	cp -r build/html/* $(GH_PAGES)
 	cd $(GH_PAGES) && \
 		git checkout gh-pages && \

source/elements/oneDAL/dalapi/extension.py

@@ -4,6 +4,7 @@
 from typing import (Dict, Tuple, Text)
 from collections import OrderedDict, namedtuple
 from . import directives
+from . import roles
 from . import doxypy
 from . import utils
 
@@ -278,6 +279,9 @@ def get_config_values(self, app):
 def setup(app):
     ctx = Context(app)
 
+    app.add_role('capterm', roles.capterm_role)
+    app.add_role('txtref', roles.txtref_role)
+
     app.add_directive('onedal_class', directives.ClassDirective(ctx))
     app.add_directive('onedal_func', directives.FunctionDirective(ctx))
     app.add_directive('onedal_code', directives.ListingDirective(ctx))

source/elements/oneDAL/dalapi/roles.py

@@ -0,0 +1,45 @@
+import re
+from docutils import nodes
+from sphinx import roles
+
+_term_ref_re = re.compile(r'(.+)<(.+)>', flags=re.DOTALL)
+def capterm_role(name, rawtext, text, lineno, inliner, options={}, content=[]):
+    xref_role = roles.XRefRole(innernodeclass=nodes.inline,
+                               warn_dangling=True)
+    term_match = _term_ref_re.match(text)
+    if term_match:
+        txt, ref = term_match.group(1), term_match.group(2)
+    else:
+        txt, ref = text, text
+    fixed_term = f'{txt.strip()} <{ref.strip().capitalize()}>'
+    return xref_role('std:term', rawtext, fixed_term, lineno, inliner, options, content)
+
+
+_term_txt_ref_re = re.compile(r'(.*)<(.+)>(.*)', flags=re.DOTALL)
+def txtref_role(name, rawtext, text, lineno, inliner, options={}, content=[]):
+    xref_role = roles.XRefRole(lowercase=True,
+                               innernodeclass=nodes.inline,
+                               warn_dangling=True)
+    def extract_ref_words(ref):
+        return [item for sub in ref.split('-') for item in sub.split('_')]
+
+    def make_term_text(words, ref, suffix=''):
+        txt = ' '.join(words).strip()
+        txt = txt[0].lower() + txt[1:]
+        return f"{txt}{suffix} <{ref.strip()}>"
+
+    term_match = _term_txt_ref_re.match(text)
+    if term_match:
+        alias, ref, suffix = term_match.group(1), term_match.group(2), term_match.group(3)
+        if len(alias) > 0 and len(suffix) == 0:
+            fixed_term = text
+        elif len(alias) == 0 and len(suffix) > 0:
+            words = extract_ref_words(ref)
+            fixed_term = make_term_text(words, ref, suffix)
+        else:
+            raise RuntimeError('Unexpected role syntax: ' + rawtext)
+    else:
+        ref, words = text, extract_ref_words(text)
+        fixed_term = make_term_text(words, ref)
+
+    return xref_role('std:ref', rawtext, fixed_term, lineno, inliner, options, content)

source/elements/oneDAL/source/algorithms/nearest_neighbors/knn_classification.rst

@@ -62,7 +62,7 @@ from the initial training set :math:`X`.
 Training method: *k-d tree*
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The training operation builds a :math:`k`-:math:`d` tree that partitions the
-training set :math:`X` (for more details, see :ref:`kd_tree`).
+training set :math:`X` (for more details, see :txtref:`k-d Tree <kd_tree>`).
 
 
 .. _i_math:

source/elements/oneDAL/source/data_management/accessors.rst

@@ -4,9 +4,9 @@
 Accessors
 =========
 
-This section defines `requirements <accessor_reqs_>`_ to an :ref:`accessor
-<accessor>` implementation and introduces several `accessor types
-<accessor_types_>`_.
+This section defines :txtref:`requirements <accessor_reqs>` to an
+:txtref:`accessor` implementation and introduces several
+:txtref:`accessor_types`.
 
 .. _accessor_reqs:
 
@@ -20,23 +20,22 @@ Each accessor implementation shall:
    accessor. Every single accessor type shall return and use only one data
    format.
 
-2. Provide an access to at least one in-memory :ref:`dataset` implementation
-   (such as :code:`table`, its sub-types, or :ref:`table builders
-   <table-builder>`).
+2. Provide an access to at least one in-memory :txtref:`dataset` implementation
+   (such as :code:`table`, its sub-types, or :txtref:`<table-builder>s`).
 
 3. Provide read-only, write-only, or read-write access to the data. If an
-   accessor supports several :ref:`dataset` implementations to be passed in, it
-   is not necessary for an accessor to support all access modes for every input
-   object. For example, tables shall support a single read-only mode according
-   to their :ref:`concept <table>` definition.
+   accessor supports several :txtref:`dataset` implementations to be passed in,
+   it is not necessary for an accessor to support all access modes for every
+   input object. For example, tables shall support a single read-only mode
+   according to their :txtref:`<table> concept` definition.
 
 4. Provide the names for read and write operations following the pattern:
 
    - :code:`pull_*()` for reading
 
    - :code:`push_*()` for writing
 
-5. Be lightweight. Its constructors from :ref:`dataset` implementations
+5. Be lightweight. Its constructors from :txtref:`dataset` implementations
    shall not have heavy operations such as copy of data, reading, writing, any
    sort of conversions. These operations shall be performed by heavy operations
    :code:`pull_*()` and :code:`push_*()`. It is not necessary to have copy- or
@@ -51,9 +50,9 @@ Accessor Types
 --------------
 
 |dal_short_name| defines a set of accessor classes. Each class is associated
-with a single specific way of interacting with data within a :ref:`dataset`. The
-following table briefly explains these classes and shows which :ref:`dataset`
-implementations are supported by each accessor type.
+with a single specific way of interacting with data within a :txtref:`dataset`.
+The following table briefly explains these classes and shows which
+:txtref:`dataset` implementations are supported by each accessor type.
 
 .. list-table::
    :header-rows: 1