Skip to content

Commit e74bcf3

Browse files
authored
Major revamp - new grey box metrics (#33)
Added new grey box metrics for evaluation for RAG without ground truth.
1 parent ca07b3d commit e74bcf3

19 files changed

+2404
-618
lines changed

.gitattributes

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
*.tar.gz filter=lfs diff=lfs merge=lfs -text

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,4 +158,9 @@ cython_debug/
158158
# and can be added to the global gitignore or merged into this file. For a more nuclear
159159
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
160160
#.idea/
161+
162+
# Ragas specific
161163
ragas/_version.py
164+
experiments/**/data
165+
experiments/**/storage
166+
**/fil-result/

experiments/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Experiments
2+
:test_tube: **This section is for experiments**
3+
4+
We commit and record results from different experiments done using Ragas here. We do two types of experiments,
5+
1. `assesments` : To develop better metrics,paradigms and evaluate metrics against human judgment.
6+
2. `baselines`: Create RAG pipelines and evaluate them using Ragas.
7+
8+
Any machine learning comprehension dataset can be used for experiments. Add any newly tried dataset to below list.
9+
Datasets:
10+
1. [HotpotQA](https://huggingface.co/datasets/hotpot_qa)
11+
2. [WikiQA test](explodinggradients/ragas-wikiqa)
12+

0 commit comments

Comments
 (0)