How to Fuse BN and ReLU with QDQ-ConvTranspose2d in TensorRT

[thfylsty]

Description

After I inserted QDQ nodes, ConvTranspose2d could no longer fuse with the subsequent BN and Relu layers, and BN and Relu would be computed in FP32 instead.

However, a model without QDQ can directly generate fused operators using --int8.

Q1: How can I make QDQ-ConvTranspose2d fuse with BN and Relu?

Q2: Can sigmoid fuse with ConvTranspose2d?

Tensorrt version 8.6.1.6

[zerollzeng]

@ttyio ^ ^

[ttyio]

This is new feature request. We have no support for Q/DQ + convTranspose + BN fusion yet.

[thfylsty]

Thank you for your reply.I can fuse bn and convtranspose manually.But is the Fusion of qdq+convT+ activation layer not supported?

This is new feature request. We have no support for Q/DQ + convTranspose + BN fusion yet.

[ttyio]

@thfylsty sorry for the delay response, yes, we should support this if this is 2D convTranspose, For 3D convTranspose, we need patterns like this: qdq + convT+ activation + q/dq

[ttyio]

Closing since no activity for more than 3 weeks, pls reopen if you still have question, thanks all!

[zzqiuzz]

@thfylsty Hi! As depicted above, i found you may utilize a tool to visualize each op perf marked with different colors.Could you please give a hint on how to achive this?