Skip to content

Commit f22e903

Browse files
authored
Docs: fix confusion regarding getting started (#815)
fixes: #290 - [x] added data preparation notebook - [x] change order and introduce flowchat : thanks to https://github.com/mtharrison - [x] Move contexts from test set to metadata
1 parent 6d38222 commit f22e903

File tree

7 files changed

+579
-15
lines changed

7 files changed

+579
-15
lines changed
405 KB
Loading
50.9 KB
Loading

docs/getstarted/evaluation.md

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33

44
Once your test set is ready (whether you've created your own or used the [synthetic test set generation module](get-started-testset-generation)), it's time to evaluate your RAG pipeline. This guide assists you in setting up Ragas as quickly as possible, enabling you to focus on enhancing your Retrieval Augmented Generation pipelines while this library ensures that your modifications are improving the entire pipeline.
55

6+
<p align="left">
7+
<img src="../_static/imgs/ragas_workflow_white.png" alt="test-outputs" width="800" height="600" />
8+
</p>
9+
610
This guide utilizes OpenAI for running some metrics, so ensure you have your OpenAI key ready and available in your environment.
711

812
```python
@@ -17,26 +21,26 @@ Let's begin with the data.
1721

1822
## The Data
1923

20-
For this tutorial, we'll use an example dataset from one of the baselines we created for the [Amnesty QA](https://huggingface.co/datasets/explodinggradients/amnesty_qa) dataset. The dataset contains the following columns:
24+
For this tutorial, we'll use an example dataset that we created using example in [data preparation](./prepare_data.ipynb). The dataset contains the following columns:
2125

2226
- question: `list[str]` - These are the questions your RAG pipeline will be evaluated on.
2327
- context: `list[list[str]]` - The contexts which were passed into the LLM to answer the question.
24-
- ground_truth: `list[str]` - The ground truth answer to the questions.
28+
- ground_truth: `str` - The ground truth answer to the questions.
29+
- answer: `str` - The answer generated by the RAG pipeline.
2530

2631
An ideal test data set should contain samples that closely mirror your real-world use case.
2732

2833
```{code-block} python
2934
:caption: import sample dataset
3035
from datasets import load_dataset
3136
32-
# loading the V2 dataset
33-
amnesty_qa = load_dataset("explodinggradients/amnesty_qa", "english_v2")
34-
amnesty_qa
37+
dataset = load_dataset("explodinggradients/prompt-engineering-guide-papers","test_data")
38+
dataset["test"]
3539
```
3640

3741
:::{seealso}
38-
See [test set generation](./testset_generation.md) to learn how to generate your own `Question/Context/Ground_Truth` triplets for evaluation.
39-
See [preparing your own dataset](/docs/howtos/applications/data_preparation.md) to learn how to prepare your own dataset for evaluation.
42+
See [test set generation](./testset_generation.md) to learn how to generate your own `Question/Ground_Truth` pairs for evaluation.
43+
See [dataset preparation](./prepare_data.ipynb) to learn how to prepare your own dataset for evaluation.
4044
:::
4145

4246
## Metrics
@@ -77,7 +81,7 @@ Running the evaluation is as simple as calling `evaluate` on the `Dataset` with
7781
from ragas import evaluate
7882
7983
result = evaluate(
80-
amnesty_qa["eval"],
84+
dataset["eval"],
8185
metrics=[
8286
context_precision,
8387
faithfulness,

docs/getstarted/index.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
:hidden:
77
install.md
88
testset_generation.md
9+
prepare_data.md
910
evaluation.md
1011
monitoring.md
1112
:::
@@ -26,7 +27,15 @@ Let's get started!
2627
:link: get-started-testset-generation
2728
:link-type: ref
2829

29-
Learn how to generate `Question/Context/Ground_Truth` triplets to get started.
30+
Learn how to generate high quality and diverse `Question/Ground_Truth` pairs to get started.
31+
:::
32+
33+
:::{card} Prepare data for evaluation
34+
:link: dataset-preparation
35+
:link-type: ref
36+
37+
Learn how to prepare a complete test dataset for evaluating using ragas metrics.
38+
3039
:::
3140

3241
:::{card} Evaluate Using Your Testset

0 commit comments

Comments
 (0)