Skip to content

Commit cfb247a

Browse files
authored
[NVBug 5659126] The same workaround for RMSNorm exporting for diffusers >=0.35.0 (#642)
## What does this PR do? **Type of change:** ? Bug fix **Overview:** ? For the trt_diffusions script ## Testing python diffusion_trt.py --model flux-dev --benchmark --skip-image Signed-off-by: Shengliang Xu <[email protected]>
1 parent 6702f99 commit cfb247a

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

examples/diffusers/quantization/diffusion_trt.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,18 @@
1818

1919
import numpy as np
2020
import torch
21+
22+
# This is a workaround for making the onnx export of models that use the torch RMSNorm work. We will
23+
# need to move on to use dynamo based onnx export to properly fix the problem. The issue has been hit
24+
# by both external users https://github.com/NVIDIA/TensorRT-Model-Optimizer/issues/262, and our
25+
# internal users from MLPerf Inference.
26+
#
27+
if __name__ == "__main__":
28+
from diffusers.models.normalization import RMSNorm as DiffuserRMSNorm
29+
30+
torch.nn.RMSNorm = DiffuserRMSNorm
31+
torch.nn.modules.normalization.RMSNorm = DiffuserRMSNorm
32+
2133
from onnx_utils.export import (
2234
_create_trt_dynamic_shapes,
2335
generate_dummy_inputs_and_dynamic_axes_and_shapes,

0 commit comments

Comments
 (0)