Skip to content

Commit 8a891f4

Browse files
authored
Merge pull request #243 from TheTom/fix/vulkan-turbo3-flat-dequant-241
vulkan: fix turbo3 signs ballot packing on wave64 subgroups
2 parents 59145a4 + 11a8377 commit 8a891f4

1 file changed

Lines changed: 11 additions & 3 deletions

File tree

ggml/src/ggml-vulkan/vulkan-shaders/copy_to_quant.comp

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -450,11 +450,19 @@ void main() {
450450
data_q[db].qs[t / 4u] = uint8_t(qs_byte);
451451
}
452452

453-
// Pack signs: 8 elements per byte via subgroup ballot
453+
// Pack signs: 8 elements per byte via subgroup ballot.
454+
// The ballot result is a uvec4 with one bit per subgroup lane, 32 lanes per
455+
// component. On wave32 every lane's bit is in ballot.x, but on wave64
456+
// (GCN/Polaris) lanes 32-63 land in ballot.y — indexing .x with a shift of
457+
// (sg_lane/8)*8 >= 32 is UB and reads the wrong lanes' bits, corrupting
458+
// half of every signs byte at write time (#241). Select the component by
459+
// lane/32 and the byte within it by (lane%32)/8, valid for any subgroup
460+
// size that is a multiple of 8.
454461
uvec4 ballot = subgroupBallot(((idx >> 2u) & 1u) != 0u);
455462
if (sg_lane % 8u == 0u) {
456-
uint local_byte = sg_lane / 8u;
457-
data_q[db].signs[t / 8u] = uint8_t((ballot.x >> (local_byte * 8u)) & 0xFFu);
463+
uint ballot_word = sg_lane / 32u;
464+
uint byte_in_word = (sg_lane % 32u) / 8u;
465+
data_q[db].signs[t / 8u] = uint8_t((ballot[ballot_word] >> (byte_in_word * 8u)) & 0xFFu);
458466
}
459467

460468
// Step 7: reconstruction norm via subgroup reduction

0 commit comments

Comments
 (0)