Commit 3cf1114
committed
Arm backend: Explicitly convert quantized value to int64
Previously, dequantizing a value with dequantize_value() in
backends/arm/tosa_quant_utils.py could result in integer overflow when
using numpy 2.1.3. The offending part of the formula is `qx - qargs.zp`.
If the subtraction results in a value outside of the range of the dtype
of `qx` the following warning is printed:
"RuntimeWarning: overflow encountered in scalar subtract"
With numpy 1.21.3 the dtype is implicitly convert to a dtype that can
store the correct value. However, in numpy 2.1.3 there's no such
conversion, leading the function to return an incorrect value.
Here's a concrete example:
```
import numpy as np
a = np.int8(127)
b = -128
print(a-b)
```
Numpy 1.21.3: a - b = 255
Numpy 2.1.3: a - b = -1
To remedy this, explicitly convert qx to int64.
Change-Id: Ie0e9e7745a424103ce650e2d58fe1a1a4cbd30e11 parent e89e320 commit 3cf1114
1 file changed
+1
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
| 74 | + | |
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| |||
0 commit comments