chore: Updates to documentation

narendasan · narendasan · commit afa97b96529c · 2021-11-08T21:11:02.000-08:00
Signed-off-by: Naren Dasan &lt;naren@narendasan.com&gt;
Signed-off-by: Naren Dasan &lt;narens@nvidia.com&gt;
diff --git a/README.md b/README.md
@@ -12,14 +12,25 @@ More Information / System Architecture:
 
 
 
-## Building a docker container for Torch-TensorRT Preview
+## Building a docker container for Torch-TensorRT
 
-We provide a `Dockerfile` in `docker/` directory. We build `Torch-TensorRT` on top of a `Pytorch NGC container` which provide basic dependencies (like CUDA, CUDNN, CUBLAS, TensorRT, Pytorch and others) The dependency libraries in the container can be found in the <a href="https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html">release notes</a>.
+We provide a `Dockerfile` in `docker/` directory. It expects a PyTorch NGC container as a base but can easily be modified to build on top of any container that provides, PyTorch, CUDA, cuDNN and TensorRT. The dependency libraries in the container can be found in the <a href="https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/index.html">release notes</a>.
 
 Please follow this instruction to build a Docker container.
 
 ```bash
-docker build -f docker/Dockerfile -t torch_tensorrt_preview:latest .
+docker build --build-arg BASE=<CONTAINER VERSION e.g. 21.11> -f docker/Dockerfile -t torch_tensorrt:latest .
+```
+
+In the case of building on top of a custom base container, you first must determine the
+version of the PyTorch C++ ABI. If your source of PyTorch is pytorch.org, likely this is the pre-cxx11-abi in which case you must modify `//docker/dist-build.sh` to not build the
+C++11 ABI version of Torch-TensorRT.
+
+You can then build the container using:
+
+
+```bash
+docker build --build-arg BASE_IMG=<IMAGE> -f docker/Dockerfile -t torch_tensorrt:latest .
 ```
 
 If you would like to build outside a docker container, please follow the section [Compiling Torch-TensorRT](#compiling-torch-tensorrt)
@@ -86,18 +97,19 @@ torch.jit.save(trt_ts_module, "trt_torchscript_module.ts") # save the TRT embedd
 | Linux aarch64 / DLA | **Native Compilation Supported on JetPack-4.4+** |
 | Windows / GPU       | **Unofficial Support**                           |
 | Linux ppc64le / GPU | -                                                |
+| NGC Containers      | **Including in PyTorch NGC Containers 21.11+**   |
 
-Torch-TensorRT will be included in NVIDIA NGC containers (https://ngc.nvidia.com/catalog/containers/nvidia:pytorch) starting in 21.11.
+> Torch-TensorRT will be included in NVIDIA NGC containers (https://ngc.nvidia.com/catalog/containers/nvidia:pytorch) starting in 21.11.
 
 > Note: Refer NVIDIA NGC container(https://ngc.nvidia.com/catalog/containers/nvidia:l4t-pytorch) for PyTorch libraries on JetPack.
 
 ### Dependencies
 
 These are the following dependencies used to verify the testcases. Torch-TensorRT can work with other versions, but the tests are not guaranteed to pass.
 
-- Bazel 4.0.0
+- Bazel 4.2.1
 - Libtorch 1.10.0 (built with CUDA 11.3)
-- CUDA 11.1 (10.2 on Jetson)
+- CUDA 11.3 (10.2 on Jetson)
 - cuDNN 8.2
 - TensorRT 8.0.3.4 (TensorRT 8.0.1.6 on Jetson)
 
@@ -147,7 +159,7 @@ bazel build //:libtorchtrt --compilation_mode opt --distdir third_party/dist_dir
 
 #### 2. Building using locally installed cuDNN & TensorRT
 
-> If you find bugs and you compiled using this method please disclose it in the issue
+> If you find bugs and you compiled using this method please disclose you used this method in the issue
 > (an `ldd` dump would be nice too)
 
 1. Install TensorRT, CUDA and cuDNN on the system before starting to compile.
diff --git a/cpp/include/torch_tensorrt/torch_tensorrt.h b/cpp/include/torch_tensorrt/torch_tensorrt.h
@@ -141,8 +141,8 @@ class TORCHTRT_API DataType {
   Value value;
 };
 
-/*
- * Setting data structure for Target device
+/**
+ * @brief Setting data structure for Target device
  */
 struct Device {
   /**
@@ -262,6 +262,10 @@ enum class EngineCapability : int8_t {
   kDLA_STANDALONE,
 };
 
+
+/**
+ * @brief TensorFormat is an enum class which defines the memeory layout used to store Tensor Data
+ * */
 class TORCHTRT_API TensorFormat {
  public:
   /**
@@ -382,8 +386,6 @@ struct TORCHTRT_API Input {
    * / traditional TRT convection (FP32 for FP32 only, FP16 for FP32 and FP16, FP32 for Int8)
    *
    * @param shape Input tensor shape
-   * @param dtype Expected data type for the input (Defaults to the type of the weights in the first tensor
-   * calculation if detectable else Float32)
    * @param format Expected tensor format for the input (Defaults to contiguous)
    */
   Input(std::vector<int64_t> shape, TensorFormat format = TensorFormat::kContiguous);
diff --git a/docsrc/index.rst b/docsrc/index.rst
@@ -43,41 +43,54 @@ Getting Started
    tutorials/use_from_pytorch
    tutorials/runtime
    tutorials/using_dla
-   _notebooks/lenet
 
 .. toctree::
    :caption: Notebooks
    :maxdepth: 1
    :hidden:
 
    _notebooks/lenet-getting-started
+   _notebooks/Resnet50-example
    _notebooks/ssd-object-detection-demo
    _notebooks/vgg-qat
 
 
 Python API Documenation
 ------------------------
 * :ref:`torch_tensorrt_py`
+* :ref:`torch_tensorrt_logging_py`
+* :ref:`torch_tensorrt_ptq_py`
+* :ref:`torch_tensorrt_ts_py`
+
 
 .. toctree::
    :caption: Python API Documenation
    :maxdepth: 0
    :hidden:
 
    py_api/torch_tensorrt
-   py_api/ts
    py_api/logging
+   py_api/ptq
+   py_api/ts
 
 C++ API Documenation
 ----------------------
 * :ref:`namespace_torch_tensorrt`
+* :ref:`namespace_torch_tensorrt__logging`
+* :ref:`namespace_torch_tensorrt__ptq`
+* :ref:`namespace_torch_tensorrt__torchscript`
+
 
 .. toctree::
    :caption: C++ API Documenation
    :maxdepth: 1
    :hidden:
 
-   _cpp_api/torch_tensorrt_cpp
+   _cpp_api/torch_tensort_cpp
+   _cpp_api/namespace_torch_tensorrt
+   _cpp_api/namespace_torch_tensorrt__logging
+   _cpp_api/namespace_torch_tensorrt__torchscript
+   _cpp_api/namespace_torch_tensorrt__ptq
 
 Contributor Documentation
 --------------------------------
diff --git a/docsrc/py_api/logging.rst b/docsrc/py_api/logging.rst
@@ -1,3 +1,5 @@
+.. _torch_tensorrt_logging_py:
+
 torch_tensorrt.logging
 ----------------------
 
diff --git a/docsrc/py_api/ptq.rst b/docsrc/py_api/ptq.rst
@@ -0,0 +1,27 @@
+.. _torch_tensorrt_ptq_py:
+
+torch_tensorrt.ptq
+----------------------
+
+.. currentmodule:: torch_tensorrt.ptq
+
+.. automodule:: torch_tensorrt.ptq
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Classes
+---------
+
+.. autoclass:: DataLoaderCalibrator
+   :members:
+   :special-members: __init__
+
+.. autoclass:: CacheCalibrator
+   :members:
+   :special-members: __init__
+
+Enums
+-------
+
+.. autoclass:: CalibrationAlgo
diff --git a/docsrc/py_api/torch_tensorrt.rst b/docsrc/py_api/torch_tensorrt.rst
@@ -1,10 +1,12 @@
 .. _torch_tensorrt_py:
 
+torch_tensorrt
+===============
+
 .. automodule torch_tensorrt
    :undoc-members:
 
-torch_tensorrt
-===============
+
 
 .. automodule:: torch_tensorrt
    :members:
@@ -53,4 +55,6 @@ Submodules
    :maxdepth: 1
 
    logging
+   ptq
    ts
+
diff --git a/docsrc/py_api/ts.rst b/docsrc/py_api/ts.rst
@@ -1,3 +1,5 @@
+.. _torch_tensorrt_ts_py:
+
 torch_tensorrt.ts
 ===================
 
diff --git a/docsrc/tutorials/installation.rst b/docsrc/tutorials/installation.rst
@@ -25,7 +25,7 @@ You can install the python package using
 
 .. code-block:: sh
 
-    pip3 install torch_tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases
+    pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases
 
 .. _bin-dist:
 
@@ -46,7 +46,7 @@ Dependencies for Compilation
 
 Torch-TensorRT is built with Bazel, so begin by installing it.
 
-    * The easiest way is to install bazelisk using the method of you choosing https://github.com/bazelbuild/bazelisk
+    * The easiest way is to install bazelisk using the method of your choosing https://github.com/bazelbuild/bazelisk
     * Otherwise you can use the following instructions to install binaries https://docs.bazel.build/versions/master/install.html
     * Finally if you need to compile from source (e.g. aarch64 until bazel distributes binaries for the architecture) you can use these instructions
 
@@ -67,9 +67,40 @@ the CUDA driver installed and the container must have CUDA)
 The correct LibTorch version will be pulled down for you by bazel.
 
     NOTE: For best compatability with official PyTorch, use torch==1.10.0+cuda113, TensorRT 8.0 and cuDNN 8.2 for CUDA 11.3 however Torch-TensorRT itself supports
-    TensorRT and cuDNN for CUDA versions other than 11.1 for usecases such as using NVIDIA compiled distributions of PyTorch that use other versions of CUDA
+    TensorRT and cuDNN for other CUDA versions for usecases such as using NVIDIA compiled distributions of PyTorch that use other versions of CUDA
     e.g. aarch64 or custom compiled version of PyTorch.
 
+.. _abis:
+
+Choosing the Right ABI
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Likely the most complicated thing about compiling Torch-TensorRT is selecting the correct ABI. There are two options
+which are incompatible with each other, pre-cxx11-abi and the cxx11-abi. The complexity comes from the fact that while
+the most popular distribution of PyTorch (wheels downloaded from pytorch.org/pypi directly) use the pre-cxx11-abi, most
+other distributions you might encounter (e.g. ones from NVIDIA - NGC containers, and builds for Jetson as well as certain
+libtorch builds and likely if you build PyTorch from source) use the cxx11-abi. It is important you compile Torch-TensorRT
+using the correct ABI to function properly. Below is a table with general pairings of PyTorch distribution sources and the
+recommended commands:
+
++-------------------------------------------------------------+----------------------------------------------------------+--------------------------------------------------------------------+
+| PyTorch Source                                              | Recommended C++ Compilation Command                      | Recommended Python Compilation Command                             |
++=============================================================+==========================================================+====================================================================+
+| PyTorch whl file from PyTorch.org                           | bazel build //:libtorchtrt -c opt --config pre_cxx11_abi | python3 setup.py bdist_wheel                                       |
++-------------------------------------------------------------+----------------------------------------------------------+--------------------------------------------------------------------+
+| libtorch-shared-with-deps-*.zip from PyTorch.org            | bazel build //:libtorchtrt -c opt --config pre_cxx11_abi | python3 setup.py bdist_wheel                                       |
++-------------------------------------------------------------+----------------------------------------------------------+--------------------------------------------------------------------+
+| libtorch-cxx11-abi-shared-with-deps-*.zip from PyTorch.org  | bazel build //:libtorchtrt -c opt                        | python3 setup.py bdist_wheel --use-cxx11-abi                       |
++-------------------------------------------------------------+----------------------------------------------------------+--------------------------------------------------------------------+
+| PyTorch preinstalled in an NGC container                    | bazel build //:libtorchtrt -c opt                        | python3 setup.py bdist_wheel --use-cxx11-abi                       |
++-------------------------------------------------------------+----------------------------------------------------------+--------------------------------------------------------------------+
+| PyTorch from the NVIDIA Forums for Jetson                   | bazel build //:libtorchtrt -c opt                        | python3 setup.py bdist_wheel --jetpack-version 4.6 --use-cxx11-abi |
++-------------------------------------------------------------+----------------------------------------------------------+--------------------------------------------------------------------+
+| PyTorch built from Source                                   | bazel build //:libtorchtrt -c opt                        | python3 setup.py bdist_wheel --use-cxx11-abi                       |
++-------------------------------------------------------------+----------------------------------------------------------+--------------------------------------------------------------------+
+
+    NOTE: For all of the above cases you must correctly declare the source of PyTorch you intend to use in your WORKSPACE file for both Python and C++ builds. See below for more information
+
 You then have two compilation options:
 
 .. _build-from-archive:
diff --git a/docsrc/tutorials/use_from_pytorch.rst b/docsrc/tutorials/use_from_pytorch.rst
@@ -1,7 +1,7 @@
 .. _use_from_pytorch:
 
 Using Torch-TensorRT Directly From PyTorch
-====================================
+============================================
 
 You will now be able to directly access TensorRT from PyTorch APIs. The process to use this feature
 is very similar to the compilation workflow described in :ref:`getting_started_with_python_api`
@@ -33,7 +33,7 @@ at the documentation for the Torch-TensorRT ``TensorRTCompileSpec`` API.
 
     spec = {
         "forward":
-            torch_tensorrt.TensorRTCompileSpec({
+            torch_tensorrt.ts.TensorRTCompileSpec({
                 "inputs": [torch_tensorrt.Input([1, 3, 300, 300])],
                 "enabled_precisions": {torch.float, torch.half},
                 "refit": False,
diff --git a/notebooks/lenet-getting-started.ipynb b/notebooks/lenet-getting-started.ipynb
diff --git a/notebooks/vgg-qat.ipynb b/notebooks/vgg-qat.ipynb
diff --git a/py/setup.py b/py/setup.py

Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,5 @@`
	`1`	`+.. _torch_tensorrt_logging_py:`
	`2`	`+`
`1`	`3`	`torch_tensorrt.logging`
`2`	`4`	`----------------------`
`3`	`5`
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,5 @@`
	`1`	`+.. _torch_tensorrt_ts_py:`
	`2`	`+`
`1`	`3`	`torch_tensorrt.ts`
`2`	`4`	`===================`
`3`	`5`