Skip to content

What is the format of json file to run on polygraphy? #4637

@JohnHerry

Description

@JohnHerry

According the code https://github.com/NVIDIA/TensorRT/blob/main/tools/Polygraphy/examples/cli/run/05_comparing_with_custom_input_data/data_loader.py
I had save the real model inference inputs as

from polygraphy.json import save_json
_res = {
    "x": x_in.detach().cpu().numpy(),
    "mask": mask_in.detach().cpu().numpy(),
    "mu": mu_in.detach().cpu().numpy(),
    "t": t_in.detach().cpu().numpy(),
    "spks": spks_in.detach().cpu().numpy(),
    "cond": cond_in.detach().cpu().numpy(),
}
save_json(_res, "real_estimator_input.json")

but the saved json sames failed to run in the polygraphy. My command is

polygraphy run --fp16 --trt --onnxrt  my_dynamic_mix_precision.onnx --trt-min-shapes  x:[2,80,576],mask:[2,1,576],mu:[2,1,576],mu:[2,80,576],cond:[2,80,576] --trt-opt-shapes x:[2,80,576],mask:[2,1,576],mu:[2,1,576],mu:[2,80,576],cond:[2,80,576] --trt-max-shapes x:[2,80,576],mask:[2,1,576],mu:[2,1,576],mu:[2,80,576],cond:[2,80,576] --load-inputs real_estimator_input.json

where the parameter of x, mask, mu, cond are of dynamic shapes.

the result:

Image

About my env:
pytorch ==2.5.1
onnx==1.16.0
onnxruntim==1.21.0
onnxruntime-gpu==1.18.0
tensorrt==9.3.0.post12.dev1
polygraphy==0.49.26

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions