You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/index.md
+68-6Lines changed: 68 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,22 +38,84 @@ Ragas is a library that provides tools to supercharge the evaluation of Large La
38
38
39
39
## Frequently Asked Questions
40
40
41
-
<divclass="toggle-list"><spanclass="arrow">▶</span> What is the best open-source model to use?</div>
41
+
<divclass="toggle-list"><spanclass="arrow">→</span> What is the best open-source model to use?</div>
42
42
<divstyle="display: none;">
43
43
There isn't a single correct answer to this question. With the rapid pace of AI model development, new open-source models are released every week, often claiming to outperform previous versions. The best model for your needs depends largely on your GPU capacity and the type of data you're working with.
44
+
<br><br>
44
45
It's a good idea to explore newer, widely accepted models with strong general capabilities. You can refer to <a href="https://github.com/eugeneyan/open-llms?tab=readme-ov-file#open-llms">this list</a> for available open-source models, their release dates, and fine-tuned variants.
45
46
</div>
46
47
47
-
<divclass="toggle-list"><spanclass="arrow">▶</span> Why do NaN values appear in evaluation results?</div>
48
+
<divclass="toggle-list"><spanclass="arrow">→</span> Why do NaN values appear in evaluation results?</div>
48
49
<divstyle="display: none;">
49
50
NaN stands for "Not a Number." In ragas evaluation results, NaN can appear for two main reasons:
<li><strong>JSON Parsing Issue:</strong> The model's output is not JSON-parsable. ragas requires models to output JSON-compatible responses because all prompts are structured using Pydantic. This ensures efficient parsing of LLM outputs.</li>
52
-
<li><strong>Non-Ideal Cases for Scoring:</strong> Certain cases in the sample may not be ideal for scoring. For example, scoring the faithfulness of a response like "I don’t know" might not be appropriate.</li>
53
+
<li><strong>Non-Ideal Cases for Scoring:</strong> Certain cases in the sample may not be ideal for scoring. For example, scoring the faithfulness of a response like "I don't know" might not be appropriate.</li>
53
54
</ul>
54
55
</div>
55
56
56
-
<divclass="toggle-list"><spanclass="arrow">▶</span> How can I make evaluation results more explainable?</div>
57
+
<divclass="toggle-list"><spanclass="arrow">→</span> How can I make evaluation results more explainable?</div>
57
58
<divstyle="display: none;">
58
59
The best way is to trace and log your evaluation, then inspect the results using LLM traces. You can follow a detailed example of this process <a href="/howtos/customizations/metrics/tracing/">here</a>.
0 commit comments