Skip to content

Commit e6a226e

Browse files
authored
[Bug] Fix AttributeError: 'QKVParallelLinear' object has no attribute 'orig_dtype' (#25958)
Signed-off-by: yewentao256 <[email protected]>
1 parent a2e6fa7 commit e6a226e

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ def create_weights(self, layer: torch.nn.Module,
6666
output_size_per_partition = sum(output_partition_sizes)
6767
layer.logical_widths = output_partition_sizes
6868
layer.weight_block_size = None
69+
layer.orig_dtype = params_dtype
6970

7071
if self.strategy == QuantizationStrategy.BLOCK:
7172
assert self.weight_block_size is not None

0 commit comments

Comments
 (0)