[quantization] is there any plan to support 6bits quantization , as 6 bits quantization more efficient than 8 bits on arm cpu

[tianylijun]

Is there any plan to support 6bits quantization , as data overflow risk, 6 bits quantization more efficient than 8 bits on ARM CPU.

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Feature

[pengzhao-intel]

Thanks for the proposal.
Do you have any technical details about 6bits is more efficient than 8bits?

[tianylijun]

Thanks for the proposal.
Do you have any technical details about 6bits is more efficient than 8bits?

As current ARM SIMD do not support int32 += int8xint8（except Cotex A55，A75)，unfortunate 8bits MAC will overflow during 4X4 sgemm block(127x127x4 exceed int16 range )，so int8 need first convert to int16 then do int32 += int16xint16. but 6 bits will not overflow in 4x4 sgemm block，so 6 bits can use directly use int16+= int8xint8 MAC，more efficient than 8 bits

[pengzhao-intel]

@tianylijun thanks for the explanation. The tensorcore compute the 4x4 GEMM with INT8 and seems it can handle the overflow. Did you have a chance to look into the difference?