Error during quantization for YOLOv10 (TopK).

[jmpark0808]

Hi there,

I recently had some success with Resnet model quantization so I wanted to move to YOLO models.
I tried quantizing (int8) a YOLOv10m model from Ultralytics after exporting it to onnx format.

y = YOLO(path/to/yolov10m.yaml)
export_path = y.export(
    format="onnx",
    half=False,
    imgsz=640,
    opset=21,
    dynamic=False,
    simplify=True,
    device="cpu",
)

Then, I tried quantizing it with this

quantize(
        onnx_path=path/to/yolo.onnx,
        quantize_mode="int8",  # fp8, int8, int4 etc.
        calibration_data=calib_tensor,
        calibration_method="max",  # max, entropy, awq_clip, rtn_dq etc.
        output_path=path/to/output.onnx,
        high_precision_dtype="fp32",
    )

but got this error

File "conda\envs\test\Lib\site-packages\onnxruntime\tools\symbolic_shape_infer.py", line 2127, in _infer_TopK
    rank = self._get_shape_rank(node, 0)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\conda\envs\test\Lib\site-packages\onnxruntime\tools\symbolic_shape_infer.py", line 417, in _get_shape_rank
    return len(self._get_shape(node, idx))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

So I dug a bit deeper, and then found out that this error got triggered on this node

input: "amax"
input: "val_172_1"
output: "topk__0"
output: "getitem_64"
name: "node_topk__1"
op_type: "TopK"
attribute {
  name: "axis"
  i: -1
  type: INT
}
attribute {
  name: "largest"
  i: 1
  type: INT
}
attribute {
  name: "sorted"
  i: 1
  type: INT
}

with this vi

name: "amax"
type {
  tensor_type {
    elem_type: 1
  }
}

where from this function, type_proto.tensor_type.HasField("shape") returned false.

def get_shape_from_type_proto(type_proto):
    assert not is_sequence(type_proto)
    if type_proto.tensor_type.HasField("shape"):
        return [get_dim_from_proto(d) for d in type_proto.tensor_type.shape.dim]
    else:
        return None  # note no shape is different from shape without dim (scalar)

How do I get around this error?

[ajrasane]

Could you once try performing shape inference of the model before the quantization process? (link)

[jmpark0808]

Hi @ajrasane,

Thank you for getting back to me.

So I tried the following

from onnx.shape_inference import infer_shapes


yolov10onnx = onnx.load(FP32_ONNX)
inferred_model = infer_shapes(yolov10onnx)
inferred_path = "yolov10m_infer.onnx"
onnx.save(inferred_model, inferred_path)

    
quantize(
        onnx_path=inferred_path,
        quantize_mode="int8",  # fp8, int8, int4 etc.
        calibration_data=calib_tensor,
        calibration_method="max",  # max, entropy, awq_clip, rtn_dq etc.
        output_path=FP8_ONNX,
        high_precision_dtype="fp32",
)

But I still get the same error

TypeError: object of type 'NoneType' has no len()