-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Closed
Labels
Module:PolygraphyIssues with PolygraphyIssues with Polygraphy
Description
According the code https://github.com/NVIDIA/TensorRT/blob/main/tools/Polygraphy/examples/cli/run/05_comparing_with_custom_input_data/data_loader.py
I had save the real model inference inputs as
from polygraphy.json import save_json
_res = {
"x": x_in.detach().cpu().numpy(),
"mask": mask_in.detach().cpu().numpy(),
"mu": mu_in.detach().cpu().numpy(),
"t": t_in.detach().cpu().numpy(),
"spks": spks_in.detach().cpu().numpy(),
"cond": cond_in.detach().cpu().numpy(),
}
save_json(_res, "real_estimator_input.json")
but the saved json sames failed to run in the polygraphy. My command is
polygraphy run --fp16 --trt --onnxrt my_dynamic_mix_precision.onnx --trt-min-shapes x:[2,80,576],mask:[2,1,576],mu:[2,1,576],mu:[2,80,576],cond:[2,80,576] --trt-opt-shapes x:[2,80,576],mask:[2,1,576],mu:[2,1,576],mu:[2,80,576],cond:[2,80,576] --trt-max-shapes x:[2,80,576],mask:[2,1,576],mu:[2,1,576],mu:[2,80,576],cond:[2,80,576] --load-inputs real_estimator_input.json
where the parameter of x, mask, mu, cond are of dynamic shapes.
the result:
About my env:
pytorch ==2.5.1
onnx==1.16.0
onnxruntim==1.21.0
onnxruntime-gpu==1.18.0
tensorrt==9.3.0.post12.dev1
polygraphy==0.49.26
Metadata
Metadata
Assignees
Labels
Module:PolygraphyIssues with PolygraphyIssues with Polygraphy
