Commit be44de8
utils: raspberrypi: ctt: Fix NaNs in lens shading tables
The problem occurs when the calculation could lead to a final row (or
column) of grid squares with no pixels in them (and hence, NaNs).
One specific case is a Pi 5 with an image width (or height) of 1364,
so that's 682 Bayer quads. To give 32 grid squares it was calculating
22 quads per cell. However, 31 * 22 = 682 leaving nothing in the final
column.
The fix is to do a rounding-down division by the number of cells minus
one, rather than a rounding-up division by the number of cells. This
turns the corner case from one where the final row/column has no
pixels to one where we don't quite cover the full image, which is how
we have to handle these cases.
Signed-off-by: David Plowman <[email protected]>1 parent 2993e79 commit be44de8
1 file changed
+4
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
127 | 127 | | |
128 | 128 | | |
129 | 129 | | |
130 | | - | |
131 | | - | |
| 130 | + | |
| 131 | + | |
132 | 132 | | |
133 | 133 | | |
134 | | - | |
| 134 | + | |
| 135 | + | |
135 | 136 | | |
136 | 137 | | |
137 | 138 | | |
| |||
0 commit comments