Skip to content

Commit 186ccfa

Browse files
Update README.md (#82)
grammar corrections. --------- Co-authored-by: Jithin James <[email protected]>
1 parent c1c3768 commit 186ccfa

File tree

1 file changed

+13
-13
lines changed

1 file changed

+13
-13
lines changed

README.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,9 @@
3939
<p>
4040
</h4>
4141

42-
ragas is a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. RAG denotes a class of LLM applications that use external data to augment the LLM’s context. There are existing tools and frameworks that help you build these pipelines but evaluating it and quantifying your pipeline performance can be hard.. This is were ragas (RAG Assessment) comes in
42+
ragas is a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. RAG denotes a class of LLM applications that use external data to augment the LLM’s context. There are existing tools and frameworks that help you build these pipelines but evaluating it and quantifying your pipeline performance can be hard... This is where ragas (RAG Assessment) comes in
4343

44-
ragas provides you with the tools based on the latest research for evaluating LLM generated text to give you insights about your RAG pipeline. ragas can be integrated with your CI/CD to provide continuous check to ensure performance.
44+
ragas provides you with the tools based on the latest research for evaluating LLM-generated text to give you insights about your RAG pipeline. ragas can be integrated with your CI/CD to provide continuous checks to ensure performance.
4545

4646
## :shield: Installation
4747

@@ -67,7 +67,7 @@ os.environ["OPENAI_API_KEY"] = "your-openai-key"
6767

6868
# prepare your huggingface dataset in the format
6969
# Dataset({
70-
# features: ['question','contexts','answer'],
70+
# features: ['question', 'contexts', 'answer'],
7171
# num_rows: 25
7272
# })
7373

@@ -81,38 +81,38 @@ If you want a more in-depth explanation of core components, check out our [quick
8181
## :luggage: Metrics
8282

8383
Ragas measures your pipeline's performance against different dimensions
84-
1. **Faithfulness**: measures the information consistency of the generated answer against the given context. If any claims made in the answer that cannot be deduced from context is penalized.
84+
1. **Faithfulness**: measures the information consistency of the generated answer against the given context. If any claims are made in the answer that cannot be deduced from context is penalized.
8585

86-
2. **Context Relevancy**: measures how relevant retrieved contexts is to the question. Ideally the context should only contain information necessary to answer the question. The presence of redundant information in the context is penalized.
86+
2. **Context Relevancy**: measures how relevant retrieved contexts are to the question. Ideally, the context should only contain information necessary to answer the question. The presence of redundant information in the context is penalized.
8787

88-
3. **Answer Relevancy**: refers to the degree to which a response directly addresses and is appropriate for a given question or context. This does not take factuality of answer into consideration but rather penalize the present of redundant information or incomplete answers given a question.
88+
3. **Answer Relevancy**: refers to the degree to which a response directly addresses and is appropriate for a given question or context. This does not take the factuality of the answer into consideration but rather penalizes the present of redundant information or incomplete answers given a question.
8989

9090
4. **Aspect Critiques**: Designed to judge the submission against defined aspects like harmlessness, correctness, etc. You can also define your own aspect and validate the submission against your desired aspect. The output of aspect critiques is always binary.
9191

92-
The final `ragas_score` is the harmonic mean of of individual metric scores.
92+
The final `ragas_score` is the harmonic mean of individual metric scores.
9393

94-
To read more about our metrics, checkout [docs](/docs/metrics.md).
94+
To read more about our metrics, check out [docs](/docs/metrics.md).
9595
## 🫂 Community
96-
If you want to get more involved with Ragas, check out our [discord server](https://discord.gg/5djav8GGNZ). It's a fun community where we geek out about LLM, Retrieval, Production issues and more.
96+
If you want to get more involved with Ragas, check out our [discord server](https://discord.gg/5djav8GGNZ). It's a fun community where we geek out about LLM, Retrieval, Production issues, and more.
9797

9898
## 🔍 Open Analytics
99-
We track very basic usage metrics to guide us to figure out what our users want, what is working and what's not. As a young startup, we have to be brutally honest about this which is why we are tracking these metrics. But as an Open Startup we open-source all the data we collect. You can read more about this [here](https://github.com/explodinggradients/ragas/issues/49). **Ragas doesnot track any information that can be used to identify you or your company**. You can take a look at exactly what we track in the [code](./src/ragas/_analytics.py)
99+
We track very basic usage metrics to guide us to figure out what our users want, what is working, and what's not. As a young startup, we have to be brutally honest about this which is why we are tracking these metrics. But as an Open Startup, we open-source all the data we collect. You can read more about this [here](https://github.com/explodinggradients/ragas/issues/49). **Ragas does not track any information that can be used to identify you or your company**. You can take a look at exactly what we track in the [code](./src/ragas/_analytics.py)
100100

101101
To disable usage-tracking you set the `RAGAS_DO_NOT_TRACK` flag to true.
102102

103103

104104
## :raising_hand_man: FAQ
105105
1. Why harmonic mean?
106106

107-
Harmonic mean penalizes extreme values. For example, if your generated answer is fully factually consistent with the context (faithfulness = 1) but is not relevant to the question (relevancy = 0), a simple average would give you a score of 0.5 but a harmonic mean will give you 0.0
107+
Harmonic-Mean penalizes extreme values. For example, if your generated answer is fully factually consistent with the context (faithfulness = 1) but is not relevant to the question (relevancy = 0), a simple average would give you a score of 0.5 but a harmonic mean will give you 0.0
108108

109109
2. How to use Ragas to improve your pipeline?
110110

111111
*"Measurement is the first step that leads to control and eventually to improvement" - James Harrington*
112112

113-
Here we assume that you already have your RAG pipeline ready. When it comes to RAG pipelines, there are mainly two parts - Retriever and generator. A change in any of this should also impact your pipelines's quality.
113+
Here we assume that you already have your RAG pipeline ready. When it comes to RAG pipelines, there are mainly two parts - Retriever and generator. A change in any of these should also impact your pipelines' quality.
114114

115-
1. First, decide one parameter that you're interested in adjusting. for example the number of retrieved documents, K.
115+
1. First, decide on one parameter that you're interested in adjusting. for example the number of retrieved documents, K.
116116
2. Collect a set of sample prompts (min 20) to form your test set.
117117
3. Run your pipeline using the test set before and after the change. Each time record the prompts with context and generated output.
118118
4. Run ragas evaluation for each of them to generate evaluation scores.

0 commit comments

Comments
 (0)