Skip to content

Commit 9afd956

Browse files
dsikkakylesayrs
andauthored
Update number of groups (#178)
* set num_groups to 1 if if < 1 * Update src/compressed_tensors/quantization/lifecycle/initialize.py Co-authored-by: Kyle Sayers <[email protected]> --------- Co-authored-by: Kyle Sayers <[email protected]>
1 parent 4da7887 commit 9afd956

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

src/compressed_tensors/quantization/lifecycle/initialize.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -172,9 +172,10 @@ def _initialize_scale_zero_point_observer(
172172
# (output_channels, 1)
173173
expected_shape = (weight_shape[0], 1)
174174
elif quantization_args.strategy == QuantizationStrategy.GROUP:
175+
num_groups = weight_shape[1] // quantization_args.group_size
175176
expected_shape = (
176177
weight_shape[0],
177-
weight_shape[1] // quantization_args.group_size,
178+
max(num_groups, 1)
178179
)
179180

180181
scale_dtype = module.weight.dtype

0 commit comments

Comments
 (0)