Skip to content

Commit 7d45270

Browse files
authored
Enable TinyLLAMAs quantization (#151)
Copy-n-paste code from 11ce176 into `quantize.py`
1 parent d8f0be6 commit 7d45270

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

quantize.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -552,6 +552,8 @@ def quantize(
552552
model = Transformer.from_name(checkpoint_path.parent.name)
553553

554554
checkpoint = torch.load(str(checkpoint_path), mmap=True, weights_only=True)
555+
if "model" in checkpoint and "stories" in str(checkpoint_path):
556+
checkpoint = checkpoint["model"]
555557
model.load_state_dict(checkpoint, assign=True)
556558
model = model.to(dtype=precision, device=device)
557559

0 commit comments

Comments
 (0)