Skip to content

Commit b5cede0

Browse files
committed
Renamed guess_verse to guess_surah
1 parent 6054458 commit b5cede0

File tree

2 files changed

+10
-10
lines changed

2 files changed

+10
-10
lines changed
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
quran-evals:
22
evals:
3-
- guess_quran_verse_name
4-
- guess_quran_verse_type
3+
- guess_quran_surah_name
4+
- guess_quran_surah_type
55
- guess_which_text_is_from_quran
66
- masked_quranic_text

evals/registry/evals/quran_eval.yaml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,22 @@
1-
guess_quran_verse_name:
2-
id: guess_quran_verse_name.dev.v0
1+
guess_quran_surah_name:
2+
id: guess_quran_surah_name.dev.v0
33
description: Tests the model's ability to guess the name of a Quranic verse.
44
metrics: [accuracy]
5-
guess_quran_verse_name.dev.v0:
5+
guess_quran_surah_name.dev.v0:
66
class: evals.elsuite.modelgraded.classify:ModelBasedClassify
77
args:
8-
samples_jsonl: quran_eval/guess_quran_verse_name.jsonl
8+
samples_jsonl: quran_eval/guess_quran_surah_name.jsonl
99
eval_type: cot_classify
1010
modelgraded_spec: simple_fact
1111

12-
guess_quran_verse_type:
13-
id: guess_quran_verse_type.dev.v0
12+
guess_quran_surah_type:
13+
id: guess_quran_surah_type.dev.v0
1414
description: Tests the model's ability to guess the type of a Quranic verse (e.g. Meccan or Medinan)
1515
metrics: [accuracy]
16-
guess_quran_verse_type.dev.v0:
16+
guess_quran_surah_type.dev.v0:
1717
class: evals.elsuite.modelgraded.classify:ModelBasedClassify
1818
args:
19-
samples_jsonl: quran_eval/guess_quran_verse_type.jsonl
19+
samples_jsonl: quran_eval/guess_quran_surah_type.jsonl
2020
eval_type: cot_classify
2121
modelgraded_spec: simple_fact
2222

0 commit comments

Comments
 (0)