[Arm] Add e2e tests for quantized rsqrt, and ensure numerical stability of the LUT based impl

[digantdesai]

🚀 The feature, motivation and pitch

TODO

Alternatives

No response

Additional context

CC @limintang , @3l1, @timhall-arm
https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-vela/-/merge_requests/19

RFC (Optional)

No response

cc @freddan80 @per @zingo @oscarandersson8218

[freddan80]

For reference:
https://github.com/pytorch/executorch/blob/main/backends/arm/_passes/insert_table_ops.py#L30-L45

It would be nice to have a representative test-vector + op in:
https://github.com/pytorch/executorch/blob/main/backends/arm/test/ops/test_rsqrt.py

[digantdesai]

I guess @3l1 was saying just creating 8b look up table is not sufficient. Let me update the issue title to reflect,
add quantized e2e tests for numerical accuracy

[freddan80]

I guess @3l1 was saying just creating 8b look up table is not sufficient. Let me update the issue title to reflect,
add quantized e2e tests for numerical accuracy

Thx! I want to understand what numerical accuracy we're aiming for. The current test do meet the standard numerical accuracy targets. What are we missing? A test vector would be great.

[Erik-Lundell]

My two cents regarding this (from an Executorch perspective) is that since we use scale and zp, it should not matter whether we use int8 och uint8 accuracy-wise. If we need increased accuracy, we need to move to an int16 table. Or am I missing something?

[digantdesai]

Apologies for being vague.

@freddan80 and I were also talking about this, and looking at regor rsqrt lowering as LUT for tflite there is a 8b LUT lowering using int32 LUT from a reference model which seems different than our 8b in ET, and there is also int16 LUT lowering.

I guess @3l1 can tell us which one would work or neither, and also help us with expected input/output tensors. I guess int16 one is something we can start with given we know we need more accuracy here.

[Erik-Lundell]

Looking into that code, it seems like the actual LUT that is added as a constant tensor is still int8, see https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-vela/-/blob/main/ethosu/regor/compiler/tflite_graph_optimiser.cpp#L2524. (the variable that is added is lut which is uint8). kRSqrtLut is only used to produce lut.Why it was done that way was probably to be 100% sure to match the reference model (though I am not sure exactly what that means since there is no int32 rsqrt in the reference_model afaik, I don't know how kRSqrtLut was produced). So int16 table still seems like the way forward for me.