You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: quickstart guide with interactive LLM and project structure (#2380)
## Changes Made
### Documentation (docs/getstarted/evals.md):
- Added `quickstart` cmd to follow through guide with an example
project.
- Updated "Custom Evaluation with LLMs" to reference DiscreteMetric from
generated code
- Replaced static examples with modern `llm_factory` API
- Changed "Using Pre-Built Metrics" to AspectCritic with modern
async/await
syntax
- Updated "Evaluating on a Dataset" to use ragas.Dataset API
### Build Configuration (mkdocs.yml):
- Made social plugin conditional: `enabled: !ENV [MKDOCS_CI, true]`
### Makefile:
- Added explicit `MKDOCS_CI=false` to serve-docs target. This avoids
social plugin error in macos if in case `cairosvg` is not found.
Copy file name to clipboardExpand all lines: README.md
+24-6Lines changed: 24 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -97,21 +97,39 @@ Available templates:
97
97
98
98
### Evaluate your LLM App
99
99
100
-
This is 5 main lines:
100
+
This is a simple example evaluating a summary for accuracy:
101
101
102
102
```python
103
-
from ragas import SingleTurnSample
104
-
from ragas.metrics import AspectCritic
103
+
import asyncio
104
+
from ragas.metrics.collections import AspectCritic
105
+
from ragas.llms import llm_factory
105
106
107
+
# Setup your LLM
108
+
llm = llm_factory("gpt-4o")
109
+
110
+
# Create a metric
111
+
metric = AspectCritic(
112
+
name="summary_accuracy",
113
+
definition="Verify if the summary is accurate and captures key information.",
114
+
llm=llm
115
+
)
116
+
117
+
# Evaluate
106
118
test_data = {
107
119
"user_input": "summarise given text\nThe company reported an 8% rise in Q3 2024, driven by strong performance in the Asian market. Sales in this region have significantly contributed to the overall growth. Analysts attribute this success to strategic marketing and product localization. The positive trend in the Asian market is expected to continue into the next quarter.",
108
120
"response": "The company experienced an 8% increase in Q3 2024, largely due to effective marketing strategies and product adaptation, with expectations of continued growth in the coming quarter.",
0 commit comments