Commit 16aa75b
Increase tolerance for numerical checking
Recent x86 kernel rounding changes lead to an error in the
single_conv model. Lowering the tolerances so tests stop
failing for now.
Seems a single rounding difference leads to a "scale" sized
difference in the tensor values. Affects ~0.5% of values
PiperOrigin-RevId: 3392732801 parent c954287 commit 16aa75b
File tree
1 file changed
+3
-1
lines changed- tensorflow_model_optimization/python/core/quantization/keras/default_8bit
1 file changed
+3
-1
lines changedLines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
175 | 175 | | |
176 | 176 | | |
177 | 177 | | |
178 | | - | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
179 | 181 | | |
180 | 182 | | |
181 | 183 | | |
| |||
0 commit comments