Skip to content

Commit 031f196

Browse files
Alex Hungalexdeucher
authored andcommitted
drm/amd/display: fix shift-out-of-bounds in CalculateVMAndRowBytes
[WHY] When PTEBufferSizeInRequests is zero, UBSAN reports the following warning because dml_log2 returns an unexpected negative value: shift exponent 4294966273 is too large for 32-bit type 'int' [HOW] In the case PTEBufferSizeInRequests is zero, skip the dml_log2() and assign the result directly. Reviewed-by: Jun Lei <[email protected]> Acked-by: Qingqing Zhuo <[email protected]> Signed-off-by: Alex Hung <[email protected]> Tested-by: Daniel Wheeler <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
1 parent 01a18aa commit 031f196

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1802,7 +1802,10 @@ static unsigned int CalculateVMAndRowBytes(
18021802
}
18031803

18041804
if (SurfaceTiling == dm_sw_linear) {
1805-
*dpte_row_height = dml_min(128, 1 << (unsigned int) dml_floor(dml_log2(PTEBufferSizeInRequests * *PixelPTEReqWidth / Pitch), 1));
1805+
if (PTEBufferSizeInRequests == 0)
1806+
*dpte_row_height = 1;
1807+
else
1808+
*dpte_row_height = dml_min(128, 1 << (unsigned int) dml_floor(dml_log2(PTEBufferSizeInRequests * *PixelPTEReqWidth / Pitch), 1));
18061809
*dpte_row_width_ub = (dml_ceil(((double) SwathWidth - 1) / *PixelPTEReqWidth, 1) + 1) * *PixelPTEReqWidth;
18071810
*PixelPTEBytesPerRow = *dpte_row_width_ub / *PixelPTEReqWidth * *PTERequestSize;
18081811
} else if (ScanDirection != dm_vert) {

0 commit comments

Comments
 (0)