(Ask) why InferInheritedType handle int8 to fp16 out?

[DeepTecher]

As you described in the docs,

InferInheritedType(info); // All Inputs inherit the type of their previous node’s outputs and sets outputs to the data type of the first input.

But in the code, it seems that you handle int8 for special case. Can you tell me why you do this?
https://github.com/openppl-public/ppl.nn/blob/252e7f27eec3976a3be48bb21f15c660cddec6af/src/ppl/nn/engines/cuda/optimizer/opt_kernel.h#L264

[Si-XU]

Int8 type should be used combining with quant scale.
We strictly use function of UnifyToOutputQuant and CopyQuantType to deal with int8 inputs.

ppl.nn/src/ppl/nn/engines/cuda/optimizer/opt_kernel.h
This statement is just in case some operations forget to handle int8 inputs.

[DeepTecher]

Okay. But why do we handle int8 input to fp16 input in the case that some operations forget to handle int8 inputs?

However, perhaps some ops may be not quant op, but support int8 input and int8 output. Like on Min/Max operation

[Si-XU]

Yes, some ops support int8 input and int8 output directlly.
However, we suggest the user to specify which ops use int8 precision via "quant.json". The rest of the operators will not save quant information and use float16 by default.