Add documentation and minor cleanup

Signed-off-by: Dheeraj Peri <[email protected]>

Author: peri044

core/conversion/conversionctx/BUILD

@@ -17,7 +17,6 @@ cc_library(
     ],
     deps = [
         "@tensorrt//:nvinfer",
-        "@tensorrt//:nvinferplugin",
         "//core/util:prelude",
     ] + select({
         ":use_pre_cxx11_abi":  ["@libtorch_pre_cxx11_abi//:libtorch"],

core/conversion/conversionctx/ConversionCtx.cpp

@@ -1,11 +1,7 @@
+#include "core/conversion/conversionctx/ConversionCtx.h"
 #include <iostream>
 #include <sstream>
-
-#include "NvInferPlugin.h"
-#include "NvInferPluginUtils.h"
-
 #include <utility>
-#include "core/conversion/conversionctx/ConversionCtx.h"
 
 namespace trtorch {
 namespace core {

core/conversion/converters/impl/batch_norm.cpp

@@ -1,6 +1,5 @@
 #include "core/conversion/converters/converters.h"
 #include "core/util/prelude.h"
-// #include "core/plugins/plugin_prelude.h"
 #include "torch/torch.h"
 
 namespace trtorch {

core/conversion/converters/impl/instance_norm.cpp

@@ -0,0 +1,40 @@
+# TRTorch Plugins
+
+A library for plugins (custom layers) used in a network. This component of TRTorch library builds a separate library called `libtrtorch_plugins.so`.
+
+On a high level, TRTorch plugin library interface does the following :
+
+- Uses TensorRT plugin registry as the main data structure to access all plugins.
+
+- Automatically registers TensorRT plugins with empty namepsace.  
+
+- Automatically registers TRTorch plugins with `"trtorch"` namespace.
+
+Here is the brief description of functionalities of each file
+
+- `plugins.h` - Provides a macro to register any plugins with `"trtorch"`  namespace.
+- `register_plugins.cpp` - Main registry class which initializes both `libnvinfer` plugins and TRTorch plugins (`Interpolate` and `Normalize`)
+- `impl/interpolate_plugin.cpp` - Core implementation of interpolate plugin. Uses pytorch kernels during execution.
+- `impl/normalize_plugin.cpp` - Core implementation of normalize plugin. Uses pytorch kernels during execution.
+
+### Converter for the plugin
+A converter basically converts a pytorch layer in the torchscript graph into a TensorRT layer (in this case a plugin layer).
+We can access a plugin via the plugin name and namespace in which it is registered.
+For example, to access the Interpolate plugin, we can use
+```
+auto creator = getPluginRegistry()->getPluginCreator("Interpolate", "1", "trtorch");
+auto interpolate_plugin = creator->createPlugin(name, &fc); // fc is the collection of parameters passed to the plugin.
+```
+
+### If you have your own plugin
+
+If you'd like to compile your plugin with TRTorch,
+
+- Add your implementation to the `impl` directory
+- Add a call `REGISTER_TRTORCH_PLUGINS(MyPluginCreator)`  to `register_plugins.cpp`. `MyPluginCreator` is the plugin creator class which creates your plugin. By adding this to `register_plugins.cpp`, your plugin will be initialized and accessible (added to TensorRT plugin registry) during the `libtrtorch_plugins.so` library loading.
+- Update the `BUILD` file with the your plugin files and dependencies.
+- Implement a converter op which makes use of your plugin.
+
+Once you've completed the above steps, upon successful compilation of TRTorch library, your plugin should be available in  `libtrtorch_plugins.so`.
+
+A sample runtime application on how to run a network with plugins can be found <a href="https://github.com/NVIDIA/TRTorch/tree/master/examples/sample_rt_app" >here</a>

core/plugins/README.md

@@ -1,6 +1,6 @@
 package(default_visibility = ["//visibility:public"])
 
-# trtorch is the dowloaded tar file
+# trtorch is the directory extracted from dowloaded release tar file
 # It has include, lib, bin directory and LICENSE file
 cc_library(
     name = "trtorch_runtime",

examples/sample_rt_app/BUILD

@@ -0,0 +1,36 @@
+# Sample application which uses TRTorch runtime library and plugin library.
+
+This sample is a demonstration on how to use TRTorch runtime library `libtrtorchrt.so` along with plugin library `libtrtorch_plugins.so`
+
+In this demo, we convert two models `ConvGelu` and `Norm` to TensorRT using TRTorch python API and perform inference using `samplertapp`. In these models, `Gelu` and `Norm` layer are expressed as plugins in the network.
+
+## Generating Torch script modules with TRT Engines
+
+The following command will generate `conv_gelu.jit` and `norm.jit` torchscript modules which contain TensorRT engines.
+
+```
+python network.py
+```
+
+## Sample runtime app
+
+The main goal is to use TRTorch runtime library `libtrtorchrt.so`, a lightweight library sufficient enough to deploy your Torchscript programs containing TRT engines.
+
+1) Download the latest release from TRTorch github repo and unpack the tar file.
+
+```
+cd examples/sample_rt_app
+// Download latest TRTorch release tar file (libtrtorch.tar.gz) from https://github.com/NVIDIA/TRTorch/releases
+tar -xvzf libtrtorch.tar.gz
+```
+
+ 2) Build and run `samplertapp`
+
+​     `samplertapp` is a binary which loads the torchscript modules `conv_gelu.jit` or `norm.jit` and runs the TRT engines on a random input using TRTorch runtime components. Checkout the `main.cpp` and `BUILD ` file for necessary code and compilation dependencies.
+
+To build and run the app
+
+```
+cd TRTorch
+bs run //examples/sample_rt_app:samplertapp $PWD/examples/sample_rt_app/norm.jit
+```