Error when loading quantized model

[wangjia184]

I have a tensorflow V2 keras model, converted into onnx model and then preprocessed and quantized. Model files can be download here

Only when I load the quantized model, it failed with Error Node (QLinearConv_token_383)'s input 0 is marked single but has an empty string in the graph

Here is python code for convertion.

def convert_to_onnx_model(file_name):
    spec = (tf.TensorSpec((1, 4, 15, 15), tf.float32, name="input"),)
    tf2onnx.convert.from_keras( renju.model, input_signature=spec, opset=17, output_path=file_name+".raw")
    shape_inference.quant_pre_process( input_model_path=file_name+".raw", output_model_path=file_name+".preprocessed")
    dr = SampleInputDataReader()
    quantize_static(
        file_name+".preprocessed",
        file_name,
        dr,
        quant_format=QuantFormat.QDQ,
        per_channel=False,
        activation_type=QuantType.QInt8,
        weight_type=QuantType.QInt8,
        optimize_model=False,
    )

When I trying to load the quantized model, CreateSession C API failed with following error.

This is an invalid model. In Node, ("QLinearConv_token_383", QLinearConv, "com.microsoft", 1) : 
("","renju_model/resblocks/res_block/batch_normalization/FusedBatchNormV3__11452:0_scale": tensor(float),
"qdq_s8_to_u8_zp_conversion_token_10": tensor(uint8),
"renju_model/resblocks/res_block/conv2d_1/Conv2D/ReadVariableOp:0_quantized": tensor(int8),
"renju_model/resblocks/res_block/conv2d_1/Conv2D/ReadVariableOp:0_scale": tensor(float),
"renju_model/resblocks/res_block/conv2d_1/Conv2D/ReadVariableOp:0_zero_point": tensor(int8),
"renju_model/resblocks/res_block/conv2d_1/BiasAdd:0_scale": tensor(float),"qdq_s8_to_u8_zp_conversion_token_12": tensor(uint8),
"renju_model/resblocks/res_block/conv2d_1/BiasAdd/ReadVariableOp:0_quantized": tensor(int32),
) 
-> ("renju_model/resblocks/res_block/conv2d_1/BiasAdd": tensor(uint8),) , 
Error Node (QLinearConv_token_383)'s input 0 is marked single but has an empty string in the graph

But I can successfully load the preprocessed model. So the problem is from quantization. Please what is the issue?

[snnn]

@yufenglee , would you please help?

[wangjia184]

the error disappears after I changed from NCHW to NHWC