I have a model that has two different inputs, can I quantize it so that each input will have different bias and scale?

[orena1]

The model is in pytorch, this is how I use the conversion, but I don't think it is working.

# Set model to Eval
conversion_model = model.eval()

# function to generate representative dataset
def representative_dataset():
    for i in range(100):
        row = df.iloc[i]
        input_a, input_b = load_inputs(row)
        output = [input_a, input_b]
        yield output


# load single sample
row = df.iloc[i]
input_a, input_b = load_inputs(row)
input_a = torch.from_numpy(input_a)
input_b = torch.from_numpy(input_b)

tfl_converter_flags = {
    "optimizations": [tf.lite.Optimize.DEFAULT],
    "representative_dataset": representative_dataset,
    "target_spec.supported_ops": [tf.lite.OpsSet.TFLITE_BUILTINS_INT8],
    "inference_input_type": tf.int8,
    "inference_output_type": tf.int8,
    "target_spec.supported_types": [tf.int8],
    "_experimental_new_quantizer": True,
    # Force per-tensor quantization to be disabled
    # "_experimental_disable_per_tensor_quantization": True,
    # Enable more aggressive quantization analysis
    "_experimental_calibrate_quantization": True,
}


tfl_fullint_model = ai_edge_torch.convert(
    conversion_model, (input_a, input_b, ), _ai_edge_converter_flags=tfl_converter_flags
)

[pkgoogle]

Hi @orena1, can you explain or show information/output why you think it's not working? Generally if the statistics are different for each of your inputs -- the quantization parameters will be different (and this propagates throughout the graph). In general we recommend you start w/ the "pure pytorch" flow of just using sample inputs and converting directly like the README example.

[orena1]

Correct, not sure what it did not work before, it is working correctly now. Thanks