Skip to content

Commit 5a15058

Browse files
authored
Update README.md (#3070)
Wrong task name: mmlu_generation doesn't non exist -> mmlu_generative is the correct one
1 parent 9fbe48c commit 5a15058

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

lm_eval/tasks/mmlu/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,11 @@ Note: The `Flan` variants are derived from [here](https://github.com/jasonwei20/
3636

3737
* `mmlu`: `Original multiple-choice MMLU benchmark`
3838
* `mmlu_continuation`: `MMLU but with continuation prompts`
39-
* `mmlu_generation`: `MMLU generation`
39+
* `mmlu_generative`: `MMLU generation`
4040

4141
MMLU is the original benchmark as implemented by Hendrycks et al. with the choices in context and the answer letters (e.g `A`, `B`, `C`, `D`) in the continuation.
4242
`mmlu_continuation` is a cloze-style variant without the choices in context and the full answer choice in the continuation.
43-
`mmlu_generation` is a generation variant, similar to the original but the LLM is asked to generate the correct answer letter.
43+
`mmlu_generative` is a generation variant, similar to the original but the LLM is asked to generate the correct answer letter.
4444

4545

4646
#### Subgroups

0 commit comments

Comments
 (0)