Skip to content

Commit 1b1c5f1

Browse files
jkbonfielddaviesrob
authored andcommitted
Fix an overflow with freq=4096 in the SSE4 decoder.
Specifically, the SIMD decoder handles this case, but the tidyup code at the end for any remainder of < 32 bytes to go didn't. Triggered by the follow example: perl -e 'print "\0"x0xff0,"AAABAA\0"'|./tests/rans4x16pr -t -o5 -c 0x0201
1 parent 45cb667 commit 1b1c5f1

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

htscodecs/rANS_static32x16pr_sse4.c

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1414,8 +1414,10 @@ unsigned char *rans_uncompress_O1_32x16_sse4(unsigned char *in,
14141414
uint32_t S = s3[l[z]][m];
14151415
unsigned char c = S & 0xff;
14161416
out[i4[z]++] = c;
1417-
R[z] = (S>>(TF_SHIFT_O1+8)) * (R[z]>>TF_SHIFT_O1) +
1418-
((S>>8) & ((1u<<TF_SHIFT_O1)-1));
1417+
int f = (S>>(TF_SHIFT_O1+8));
1418+
if (f == 0)
1419+
f = 4096;
1420+
R[z] = f * (R[z]>>TF_SHIFT_O1) + ((S>>8) & ((1u<<TF_SHIFT_O1)-1));
14191421
RansDecRenormSafe(&R[z], &ptr, ptr_end);
14201422
l[z] = c;
14211423
}

0 commit comments

Comments
 (0)