Mask the high half when decoding SAP pairs to avoid a negative index - #3539
Mask the high half when decoding SAP pairs to avoid a negative index#3539Abhayindia wants to merge 1 commit into
Conversation
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
Masking the high half fixes the negative decode, but the packed value is still created with signed (id1 << 16), which is undefined once id1 >= 0x8000. Please make the packed representation unsigned end-to-end rather than only masking the decode.
c116149 to
2327d32
Compare
|
good call. reworked it: the pair buffers, |
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
Rechecked the current head. Pair storage and sort buffers are unsigned end to end, and packing casts before the shift, so IDs at or above 0x8000 no longer invoke signed-shift undefined behaviour. My previous blocker is resolved.
Fixes #3535.
mj_SAPpacks a pair of item indices as(id1<<16) + id2into a signedint. Onceid1 >= 0x8000that sets bit 31, so the packed value is negative. Two sites decode the high half with a bare arithmetic shift:so the shift sign-extends and the array is indexed at a negative offset - an out-of-bounds read (SEGV in the flex path once the garbage id feeds unchecked pointer arithmetic). A third site decoding the same packing already masks it:
This brings the other two in line with
& 0xFFFF. Reachable with a 2D flex over ~33k elements andselfcollide="auto", or a plain model with more than 32768 mutually-collidable bodies (mj_SAPonly rejectsn >= 0x10000).No unit test: triggering it needs >32768 collidable bodies, and a model that size takes minutes to compile. The issue has a full AddressSanitizer trace of the pre-fix crash.