Skip to content

Commit e28ec3a

Browse files
Merge pull request #132 from foundation-model-stack/chichun-charlie-liu-patch-1
fix: Update GPTQ example README.md for typo
2 parents 957173c + 25891cb commit e28ec3a

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

examples/GPTQ/README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ This end-to-end example utilizes the common set of interfaces provided by `fms_m
7272
7373
```bash
7474
lm_eval --model hf \
75-
--model_args pretrained="Meta-Llama-3-8B-GPTQ,dtype=float16,autogptq=True,enforce_eager=True" \
75+
--model_args pretrained="Meta-Llama-3-8B-GPTQ,dtype=float16,gptqmodel=True=True,enforce_eager=True" \
7676
--tasks lambada_openai \
7777
--num_fewshot 5 \
7878
--device cuda:0 \
@@ -82,18 +82,21 @@ This end-to-end example utilizes the common set of interfaces provided by `fms_m
8282
## Example Test Results
8383
8484
- Unquantized Model
85+
-
8586
|Model | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
8687
|------------|--------------|------:|------|-----:|----------|---|-----:|---|-----:|
8788
| LLAMA3-8B |lambada_openai| 1|none | 5|acc |↑ |0.7103|± |0.0063|
8889
| | | |none | 5|perplexity|↓ |3.7915|± |0.0727|
8990
9091
- Quantized model with the settings showed above (`desc_act` default to False.)
92+
-
9193
|Model | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
9294
|------------|--------------|------:|------|-----:|----------|---|------:|---|-----:|
9395
| LLAMA3-8B |lambada_openai| 1|none | 5|acc |↑ |0.6365 |± |0.0067|
9496
| | | |none | 5|perplexity|↓ |5.9307 |± |0.1830|
9597
9698
- Quantized model with `desc_act` set to `True` (could improve the model quality, but at the cost of inference speed.)
99+
-
97100
|Model | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
98101
|------------|--------------|------:|------|-----:|----------|---|------:|---|-----:|
99102
| LLAMA3-8B |lambada_openai| 1|none | 5|acc |↑ |0.6193 |± |0.0068|

0 commit comments

Comments
 (0)