Skip to content

Commit c372e52

Browse files
committed
Fix defects in README
Signed-off-by: Bill Murdock <[email protected]>
1 parent a946e98 commit c372e52

File tree

1 file changed

+14
-14
lines changed

1 file changed

+14
-14
lines changed

notebooks/evaluation/README.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -10,36 +10,36 @@ The primary goal is to offer a reproducible methodology for comparing RAG system
1010

1111
## Table of Contents
1212

13-
- [Project Structure](https://www.google.com/search?q=%23project-structure)
14-
- [Getting Started](https://www.google.com/search?q=%23getting-started)
15-
- [Summary of Findings](https://www.google.com/search?q=%23summary-of-findings)
16-
- [Detailed Results](https://www.google.com/search?q=%23detailed-results)
17-
- [Key Limitations of this Study](https://www.google.com/search?q=%23key-limitations-of-this-study)
18-
- [Further Observations](https://www.google.com/search?q=%23further-observations)
13+
- [Project Structure](#project-structure)
14+
- [Getting Started](#getting-started)
15+
- [Summary of Findings](#summary-of-findings)
16+
- [Detailed Results](#detailed-results)
17+
- [Key Limitations of this Study](#key-limitations-of-this-study)
18+
- [Further Observations](#further-observations)
1919

2020
## Project Structure
2121

2222
This directory includes the following components:
2323

2424
* **Jupyter Notebooks**:
2525

26-
* [`make-sample-questions.ipynb`](https://www.google.com/search?q=./make-sample-questions.ipynb): Generates a dataset of sample questions and reference answers from a source document.
27-
* [`evaluate-using-sample-questions-lls-vs-li.ipynb`](https://www.google.com/search?q=./evaluate-using-sample-questions-lls-vs-li.ipynb): Runs Llama Stack and LlamaIndex RAG pipelines on the generated questions, evaluates their responses using the Ragas framework, and performs statistical significance testing with SciPy.
26+
* [`make-sample-questions.ipynb`](./make-sample-questions.ipynb): Generates a dataset of sample questions and reference answers from a source document.
27+
* [`evaluate-using-sample-questions-lls-vs-li.ipynb`](/evaluate-using-sample-questions-lls-vs-li.ipynb): Runs Llama Stack and LlamaIndex RAG pipelines on the generated questions, evaluates their responses using the Ragas framework, and performs statistical significance testing with SciPy.
2828

2929
* **Supporting Code**:
3030

31-
* [`evaluation_utilities.py`](https://www.google.com/search?q=./evaluation_utilities.py): Utility functions and helper code for the evaluation notebooks.
31+
* [`evaluation_utilities.py`](./evaluation_utilities.py): Utility functions and helper code for the evaluation notebooks.
3232

3333
* **Sample Data**:
3434

35-
* [`qna-ibm-2024-2250-2239.json`](https://www.google.com/search?q=./qna-ibm-2024-2250-2239.json): A Q\&A dataset generated from the IBM 2024 annual report without special instructions.
36-
* [`qna-ibm-2024b-2220-2196.json`](https://www.google.com/search?q=./qna-ibm-2024b-2220-2196.json): A Q\&A dataset generated from the same report, but using the default special instructions in the notebook to produce more diverse questions.
35+
* [`qna-ibm-2024-2250-2239.json`](./qna-ibm-2024-2250-2239.json): A Q\&A dataset generated from the IBM 2024 annual report without special instructions.
36+
* [`qna-ibm-2024b-2220-2196.json`](./qna-ibm-2024b-2220-2196.json): A Q\&A dataset generated from the same report, but using the default special instructions in the notebook to produce more diverse questions.
3737
* **Note on filenames**: The numbers in the JSON filenames (`{configured_questions}-{final_question_count}`) may not perfectly match the final counts in the file due to de-duplication steps.
3838

3939
* **Configuration**:
4040

41-
* [`requirements.txt`](https://www.google.com/search?q=./requirements.txt): A list of Python libraries required to run the notebooks.
42-
* [`run.yaml`](https://www.google.com/search?q=./run.yaml): A configuration file for the Llama Stack server.
41+
* [`requirements.txt`](./requirements.txt): A list of Python libraries required to run the notebooks.
42+
* [`run.yaml`](./run.yaml): A configuration file for the Llama Stack server.
4343

4444
## Getting Started
4545

@@ -66,7 +66,7 @@ llama stack run run.yaml --image-type venv
6666
1. **(Optional)** Run `make-sample-questions.ipynb` if you want to generate your own question-answer dataset from a new document.
6767
2. Run `evaluate-using-sample-questions-lls-vs-li.ipynb` to execute the comparison between Llama Stack and LlamaIndex using one of the sample `.json` files.
6868

69-
> **Note on Scale**: Both notebooks are configured by default to run on a small number of questions for quick results. Instructions are included within the notebooks on how to adjust the configuration to run on the full datasets.
69+
> **Note on Scale**: Both notebooks are configured by default to run on a limited number of questions for quick results. Instructions are included within the notebooks on how to adjust the configuration to run on the full datasets.
7070
7171
## Summary of Findings
7272

0 commit comments

Comments
 (0)