We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent ed80e3b commit 001e5efCopy full SHA for 001e5ef
examples/models/llama/README.md
@@ -398,7 +398,7 @@ LLAMA_CHECKPOINT=path/to/checkpoint.pth
398
LLAMA_PARAMS=path/to/params.json
399
400
# Set low-bit quantization parameters
401
-QLINEAR_BITWIDTH=3 # Can be 1-8
+QLINEAR_BITWIDTH=4 # Can be 1-8
402
QLINEAR_GROUP_SIZE=128 # Must be multiple of 16
403
QEMBEDDING_BITWIDTH=4 # Can be 1-8
404
QEMBEDDING_GROUP_SIZE=32 # Must be multiple of 16
0 commit comments