Skip to content

Commit d9f0ecc

Browse files
committed
TP: Fix vocab split for models with odd vocab sizes
1 parent 7319b6e commit d9f0ecc

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

exllamav2/tensor_p.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ def define_split(
155155

156156
# Vocab split
157157

158-
vc_split = [s * 32 for s in integer_split(cfg.vocab_size // 32, gpu_split, 16)]
158+
vc_split = [s * 32 for s in integer_split((cfg.vocab_size + 31) // 32, gpu_split, 16)]
159159

160160
def set_split(raw_split):
161161
b = 0

0 commit comments

Comments
 (0)