[Bug] inconsistent shapes of results produced by TVM and ONNXRuntime due to the ConvTranspose operator

[coffezhou]

Expected behavior

The ConvTranspose operator in TVM should produce right shape.

Actual behavior

For the following model,

it can be executed by onnxruntime and onnx's ReferenceEvaluator, the shapes of results are as follows:

onnxruntime: (1, 6, 56, 56)
ReferenceEvaluator: (1, 6, 56, 56)

However, the shape of results produced by TVM is:

TVM: (1, 6, 55, 55)

which is different from those of onnxruntime and onnx's ReferenceEvaluator.

According to the documents of ConvTranspose, the shape of the output is calculated via the following equation:

output_shape[i] = stride[i] * (input_size[i] - 1) + output_padding[i] + ((kernel_shape[i] - 1) * dilations[i] + 1) - pads[start_i] - pads[end_i]

For the last dim, the shape should be:
2*(28-1) + 1 + ((3-1) + 1) - 1 - 1 = 54 + 1 + 3 - 1 - 1 = 56

This is confusing that why the shape of TVM's results is (1, 6, 55, 55).

Environment

OS: Ubuntu 20.04
TVM: 0.23.dev0 (f4e28d3)

onnxruntime: 1.23.2

Steps to reproduce

This bug can be reproduced by the following code with the model in the attachment.

from typing import Dict, List, Literal, Optional
import sys
import os

import numpy as np
import onnx
from onnx.reference import ReferenceEvaluator

import onnxruntime
from onnx import ModelProto, TensorProto, helper

import tvm
import tvm.testing
from tvm import relax
from tvm.relax.frontend.onnx import from_onnx

import argparse
import pickle

def test() -> None:
    onnx_model = onnx.load("11.onnx")
    # Configure model format.
    onnx_model.ir_version = 8
    onnx_model.opset_import[0].version = 14
    
    with open("inputs.pkl", 'rb') as fp:
        inputs = pickle.load(fp)
    # Run the model through onnx to get the expected result.
    try:
        ort_session = onnxruntime.InferenceSession(
            onnx_model.SerializeToString(), providers=["CPUExecutionProvider"]
        )
        ort_output = ort_session.run([], inputs)
    except Exception as e:
        print(e)
        print("This model cannot be executed by onnxruntime!")
        sys.exit(1)

    print("onnxruntime:", ort_output[0].shape)

    # ReferenceEvaluator
    sess = ReferenceEvaluator("11.onnx")
    re_output = sess.run(None, inputs)
    print("ReferenceEvaluator:", re_output[0].shape)

    tvm.testing.assert_allclose(re_output[0], ort_output[0], rtol=0.1, atol=0.1)

    # TVM
   # Convert the onnx model into relax through the onnx importer.
    tvm_model = from_onnx(onnx_model, opset=14, keep_params_in_input=True)
    # Convert operators for inference mode.
    tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model)
    # Legalize any relax ops into tensorir.
    tvm_model = relax.transform.LegalizeOps()(tvm_model)

    # Separate model from parameters.
    tvm_model, params = relax.frontend.detach_params(tvm_model)
    # Compile the relax graph into a VM then run.
    with tvm.transform.PassContext(opt_level=3):
        ex = tvm.compile(tvm_model, target="llvm")
        vm = relax.VirtualMachine(ex, tvm.cpu())
    # Prepare inputs.
    input_list = [
        inputs[key.name_hint] for key in tvm_model["main"].params if key.name_hint in inputs
    ]
    if params:
        input_list += params["main"]

    # Run model and check outputs.
    vm.set_input("main", *input_list)
    vm.invoke_stateful("main")
    tvm_output = vm.get_outputs("main")

    print("TVM:", tvm_output.shape)    
    
if __name__ == "__main__":
    test()
 

testcase.zip

Triage

Please refer to the list of label tags here to find the relevant tags and add them below in a bullet format (example below).

• needs-triage