Commit f6c6909
authored
Relax XNN quantized conv1d test tolerances (#15666)
### Summary
The test_qs8_conv1d_batchnorm_seq test is periodically failing in CI due
to being out of tolerance. From looking at the numbers, I believe this
is due to an overly tight tolerance when quantizing. The error reported
in
https://github.com/pytorch/executorch/actions/runs/19147331216/job/54728231147#step:9:19945,
for example, is 0.03, which seems large, but given that the output range
is 6, that's roughly 1/200th. With 8-bit quant, that's not unreasonable.
I've updated the tolerance in the test to reflect this.1 parent 6014129 commit f6c6909
1 file changed
+3
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
126 | 126 | | |
127 | 127 | | |
128 | 128 | | |
129 | | - | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
130 | 132 | | |
131 | 133 | | |
132 | 134 | | |
| |||
0 commit comments