File tree Expand file tree Collapse file tree 2 files changed +10
-10
lines changed Expand file tree Collapse file tree 2 files changed +10
-10
lines changed Original file line number Diff line number Diff line change 11quran-evals :
22 evals :
3- - guess_quran_verse_name
4- - guess_quran_verse_type
3+ - guess_quran_surah_name
4+ - guess_quran_surah_type
55 - guess_which_text_is_from_quran
66 - masked_quranic_text
Original file line number Diff line number Diff line change 1- guess_quran_verse_name :
2- id : guess_quran_verse_name .dev.v0
1+ guess_quran_surah_name :
2+ id : guess_quran_surah_name .dev.v0
33 description : Tests the model's ability to guess the name of a Quranic verse.
44 metrics : [accuracy]
5- guess_quran_verse_name .dev.v0 :
5+ guess_quran_surah_name .dev.v0 :
66 class : evals.elsuite.modelgraded.classify:ModelBasedClassify
77 args :
8- samples_jsonl : quran_eval/guess_quran_verse_name .jsonl
8+ samples_jsonl : quran_eval/guess_quran_surah_name .jsonl
99 eval_type : cot_classify
1010 modelgraded_spec : simple_fact
1111
12- guess_quran_verse_type :
13- id : guess_quran_verse_type .dev.v0
12+ guess_quran_surah_type :
13+ id : guess_quran_surah_type .dev.v0
1414 description : Tests the model's ability to guess the type of a Quranic verse (e.g. Meccan or Medinan)
1515 metrics : [accuracy]
16- guess_quran_verse_type .dev.v0 :
16+ guess_quran_surah_type .dev.v0 :
1717 class : evals.elsuite.modelgraded.classify:ModelBasedClassify
1818 args :
19- samples_jsonl : quran_eval/guess_quran_verse_type .jsonl
19+ samples_jsonl : quran_eval/guess_quran_surah_type .jsonl
2020 eval_type : cot_classify
2121 modelgraded_spec : simple_fact
2222
You can’t perform that action at this time.
0 commit comments