IPluginCreator not found in Plugin Registry error when deserializing engine model

[huanghaifeng1234]

Description

I train yolov5 using pytorch, and convert it to tensorrt engine file. When converting I use REGISTER_TENSORRT_PLUGIN(YoloPluginCreator) to register my custom plugin. When loading the engine file, I try to deserialize it but failed, below is the error infomation:

[04/11/2022-20:45:35] [E] [TRT] 1: [pluginV2Runner.cpp::nvinfer1::rt::load::290] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[04/11/2022-20:45:35] [E] [TRT] 4: [runtime.cpp::nvinfer1::Runtime::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)

I have added
initLibNvInferPlugins(&gLogger, ""); before serialization but that doesn't work for me.

Environment

TensorRT Version: 7.2.3.4
NVIDIA GPU: GeForce GTX 1660 Ti
NVIDIA Driver Version: 496.49
CUDA Version: 11.0
CUDNN Version: 8.1.1
Operating System: windows11
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 1.11.0
Baremetal or Container (if so, version):

Relevant Files

Steps To Reproduce

serialize:

deserialize:

[Yautoi]

Is the problem solved please?

[xiaoxiaozengzen]

Has the problem been solved?

I encountered the same problem and reported the following error:
:InstanceNormalization_TRT version 1Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.1: [pluginV2Runner.cpp::load::290] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)4: [runtime.cpp::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)

[acburigo]

Similar problem here. How have you solved it?

[qihao-plus]

Met the same issue:

[pluginV2Runner.cpp::load::290] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
....
ERROR 28434 tensorrt_utils.cpp:91 instanceNormalizationPlugin/instanceNormalizationPlugin.cu (171) - Cuda Error in terminate: 700 (an illegal memory access was encountered)

[oreo-lp]

Me too!

[KeyLD]

Any luck? I have similar problem when using custom plugin, but the difference is I use trtexec to build and infer.

[08/25/2022-16:42:25] [V] [TRT] Engine Layer Information:
Layer(PluginV2): RepeatInterleave_0, Tactic: 0x0000000000000000, 0[Float(3,3)] -> 56[Float(3,9)]
[08/25/2022-16:42:25] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[08/25/2022-16:42:25] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[08/25/2022-16:42:25] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[08/25/2022-16:42:25] [I] Engine built in 1.19205 sec.
[08/25/2022-16:42:25] [I] [TRT] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 619, GPU 562 (MiB)
[08/25/2022-16:42:25] [I] [TRT] Loaded engine size: 0 MiB
[08/25/2022-16:42:25] [E] Error[1]: [pluginV2Runner.cpp::load::293] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[08/25/2022-16:42:25] [E] Error[4]: [runtime.cpp::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)
[08/25/2022-16:42:25] [E] Engine deserialization failed

[LiberiFatali]

Anyone has solved this?

[HarrySm]

@LiberiFatali @KeyLD @acburigo @oreo-lp @xiaoxiaozengzen @qihao-plus @huanghaifeng1234 @Yautoi
Anyone has solved this problem. I have the exact same problem when using trtexec

[qihao-plus]

Forget to update.
I solved it by lifting the position of ”initLibNvInferPlugins“ stuff ahead, and it works.

[HarrySm]

Forget to update. I solved it by lifting the position of ”initLibNvInferPlugins“ stuff ahead, and it works.

@qihao-plus
Thank you for your reply, do you know or have any idea how to solve it in trtexec.

Thank you :)

[qihao-plus]

Forget to update. I solved it by lifting the position of ”initLibNvInferPlugins“ stuff ahead, and it works.

@qihao-plus Thank you for your reply, do you know or have any idea how to solve it in trtexec.

Thank you :)

trtexec supports tensorrt_plugin, you can check the help function.

[HarrySm]

@qihao-plus Okay thank you very much. One more question do you have the plugin or the (.so) file so I can add it to trtexec if not how you can create the plugin.

[qihao-plus]

$ trtexec --help
" --plugins Plugin library (.so) to load (can be specified multiple times)"

The lib comes from two ways:

1. the TensorRT itself provides lots of plugin libs in https://developer.nvidia.com/nvidia-tensorrt-8x-download
2. you can compile it, and add your self-implemented plugin lib together with TRT-OSS

[HarrySm]

@qihao-plus , Thank you very much I will this :)