Skip to content
This repository was archived by the owner on Mar 14, 2025. It is now read-only.

Commit b65c5de

Browse files
committed
Add a base README for list_plugins tool
1 parent c2cbbd7 commit b65c5de

File tree

3 files changed

+96
-1
lines changed

3 files changed

+96
-1
lines changed

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,8 @@ resnet50/
77
# TensorRT Engines
88
*.plan
99
*.engine
10+
11+
# Plugins
12+
*.d
13+
*.o
14+
*.so

plugins/README.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Plugins
2+
3+
This directory contains some tools and samples related to
4+
TensorRT Plugins.
5+
6+
## Listing Registered Plugins
7+
8+
```
9+
$ python list_plugins.py -h
10+
usage: list_plugins.py [-h] [-p [PLUGINS [PLUGINS ...]]]
11+
12+
Script to list registered TensorRT plugins. Can optionally load custom plugin
13+
libraries.
14+
15+
optional arguments:
16+
-h, --help show this help message and exit
17+
-p [PLUGINS [PLUGINS ...]], --plugins [PLUGINS [PLUGINS ...]]
18+
Path to a plugin (.so) library file. Accepts multiple
19+
arguments.
20+
```
21+
22+
Default `TRT_RELEASE` plugins:
23+
```
24+
$ python list_plugins.py
25+
2020-03-13 22:12:35 - __main__ - INFO - Registering plugins...
26+
2020-03-13 22:12:35 - __main__ - INFO - Registered Plugin Names:
27+
['RnRes2Br2bBr2c_TRT',
28+
'RnRes2Br1Br2c_TRT',
29+
'CgPersistentLSTMPlugin_TRT',
30+
'SingleStepLSTMPlugin',
31+
...
32+
'SpecialSlice_TRT',
33+
'InstanceNormalization_TRT']
34+
```
35+
36+
## Registering OSS Plugins
37+
38+
When building the OSS components, the default plugin library may be overwritten
39+
or given precendence to the OSS plugin library. As long as you have both .so
40+
library files, you can load both:
41+
42+
Build OSS Components:
43+
```
44+
$ wget https://raw.githubusercontent.com/rmccorm4/tensorrt-utils/master/OSS/build_OSS.sh
45+
$ source build_OSS.sh
46+
```
47+
48+
Load and list multiple plugin libraries:
49+
```
50+
$ python list_plugins.py --plugins TensorRT/build/out/libnvinfer_plugin.so
51+
2020-03-13 22:18:51 - __main__ - INFO - Loading plugin library: /mnt/TensorRT/build/out/libnvinfer_plugin.so
52+
2020-03-13 22:18:51 - __main__ - INFO - Registering plugins...
53+
2020-03-13 22:18:51 - __main__ - INFO - Registered Plugin Names:
54+
['RnRes2Br2bBr2c_TRT',
55+
'RnRes2Br1Br2c_TRT',
56+
'CgPersistentLSTMPlugin_TRT',
57+
'SingleStepLSTMPlugin',
58+
'CustomEmbLayerNormPluginDynamic', <------ OSS Plugins added
59+
'CustomFCPluginDynamic', <------ OSS Plugins added
60+
'CustomGeluPluginDynamic', <------ OSS Plugins added
61+
'CustomQKVToContextPluginDynamic', <------ OSS Plugins added
62+
'CustomSkipLayerNormPluginDynamic', <------ OSS Plugins added
63+
...
64+
'SpecialSlice_TRT',
65+
'InstanceNormalization_TRT']
66+
```
67+
68+
## Registering Custom Plugins
69+
70+
Same concept as above for OSS plugins, just giving an example to be extra clear:
71+
```
72+
$ pushd CustomIPluginV2/
73+
$ make
74+
g++ -g -std=c++11 -DNDEBUG -fPIC -MD -MP -I. -I/usr/local/cuda/include -I/usr/src/tensorrt/include -o CustomPlugin.o -c CustomPlugin.cpp
75+
g++ -g -std=c++11 -DNDEBUG -shared -o CustomPlugin.so CustomPlugin.o -L/usr/local/cuda/lib64 -L/usr/src/tensorrt/lib -lnvinfer -lcudart
76+
77+
$ popd
78+
$ python list_plugins.py --plugins CustomIPluginV2/CustomPlugin.so
79+
2020-03-13 22:16:06 - __main__ - INFO - Loading plugin library: CustomIPluginV2/CustomPlugin.so
80+
Plugin attribute number: 5
81+
2020-03-13 22:16:06 - __main__ - INFO - Registering plugins...
82+
2020-03-13 22:16:06 - __main__ - INFO - Registered Plugin Names:
83+
['RnRes2Br2bBr2c_TRT',
84+
'RnRes2Br1Br2c_TRT',
85+
'CgPersistentLSTMPlugin_TRT',
86+
'SingleStepLSTMPlugin',
87+
'CustomPlugin', <------------- Custom plugin registered
88+
...
89+
'SpecialSlice_TRT',
90+
'InstanceNormalization_TRT']
91+
```

plugins/list_plugins.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,6 @@ def get_all_plugin_names(plugin_registry):
6363
parser.add_argument("-p", "--plugins", nargs="*", default=[], help="Path to a plugin (.so) library file. Accepts multiple arguments.")
6464
args = parser.parse_args()
6565

66-
print(args.plugins)
6766
for plugin_library in args.plugins:
6867
# Example default plugin library: "/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so"
6968
logger.info("Loading plugin library: {}".format(plugin_library))

0 commit comments

Comments
 (0)