Skip to content

Commit 9e2b530

Browse files
Merge branch 'main' into kylesayrs/transform-merge
2 parents d00675c + b2df366 commit 9e2b530

File tree

1 file changed

+6
-1
lines changed
  • src/compressed_tensors/quantization/lifecycle

1 file changed

+6
-1
lines changed

src/compressed_tensors/quantization/lifecycle/forward.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,8 +124,13 @@ def dequantize(
124124
strategy=QuantizationStrategy.GROUP, group_size=group_size
125125
)
126126
else:
127+
rows, cols = x_q.shape[-2], x_q.shape[-1]
128+
block_height = rows // scale.shape[0] # Rows per block
129+
block_width = cols // scale.shape[1] # Columns per block
130+
127131
args = QuantizationArgs(
128-
strategy=QuantizationStrategy.BLOCK, block_structure=scale.shape
132+
strategy=QuantizationStrategy.BLOCK,
133+
block_structure=[block_height, block_width],
129134
)
130135
else:
131136
raise ValueError(

0 commit comments

Comments
 (0)