Skip to content

Commit 83d25e3

Browse files
authored
Remove experimental from docs and fix examples in docs (#2270)
### Summary Remove “Experimental” docs and migrate the hello-world content into a core Get Started tutorial. ### Changes - Added `docs/getstarted/experiments_quickstart.md` (tutorial: Run your first experiment). - Moved assets to `docs/_static/imgs/experiments_quickstart/` and updated image links to `/_static/...`. - Updated `mkdocs.yml`: removed `🧪 Experimental` section; added “Run your first experiment” under Get Started. - Linked from `docs/getstarted/index.md` to the new tutorial. - Cross-linked from `docs/concepts/experimentation.md` to the new tutorial. - Removed the entire `docs/experimental/` directory. ### Verification - Built docs locally succeeds. - New tutorial renders; Experimental tab is gone; links and assets resolve.
1 parent c78688b commit 83d25e3

File tree

12 files changed

+45
-82
lines changed

12 files changed

+45
-82
lines changed

docs/experimental/hello_world.gif renamed to docs/_static/imgs/experiments_quickstart/hello_world.gif

File renamed without changes.

docs/experimental/output_first_experiment.png renamed to docs/_static/imgs/experiments_quickstart/output_first_experiment.png

File renamed without changes.

docs/concepts/experimentation.md

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ graph LR
3636

3737
## Creating Experiments with Ragas
3838

39-
Ragas provides an `@experiment` decorator to streamline the experiment creation process:
39+
Ragas provides an `@experiment` decorator to streamline the experiment creation process. If you prefer a hands-on intro first, see [Run your first experiment](../getstarted/experiments_quickstart.md).
4040

4141
### Basic Experiment Structure
4242

@@ -149,17 +149,6 @@ return {
149149
}
150150
```
151151

152-
### 4. Comparative Analysis
153-
154-
Use the built-in comparison tools to analyze results:
155-
156-
```python
157-
# Compare two experiments
158-
comparison_results = compare_experiments(
159-
["experiments/exp1.csv", "experiments/exp2.csv"]
160-
)
161-
```
162-
163152
## Advanced Experiment Patterns
164153

165154
### A/B Testing

docs/experimental/core_concepts/index.md

Lines changed: 0 additions & 3 deletions
This file was deleted.

docs/experimental/howtos/index.md

Lines changed: 0 additions & 5 deletions
This file was deleted.
Lines changed: 31 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,74 +1,55 @@
1-
# Ragas Experimental
1+
# Run your first experiment
22

3-
A framework for applying Evaluation-Driven Development (EDD) to AI applications.
3+
This tutorial walks you through running your first experiment with Ragas using the `@experiment` decorator and a local CSV backend.
44

5-
The goal of Ragas Experimental is to evolve Ragas into a general-purpose evaluation framework for AI applications. It helps teams design, run, and reason about evaluations across any AI workflow. Beyond tooling, it provides a mental model for thinking about evaluations not just as a diagnostic tool, but as the backbone of iterative improvement.
6-
7-
# ✨ Introduction
8-
9-
10-
<div class="grid cards" markdown>
11-
12-
- 🚀 **Tutorials**
13-
14-
Step-by-step guides to help you get started with Ragas Experimental. Learn how to evaluate AI applications like RAGs and agents with practical examples.
15-
16-
[:octicons-arrow-right-24: Tutorials](tutorials/index.md)
17-
18-
- 📚 **Core Concepts**
19-
20-
A deeper dive into the principles of evaluation and how Ragas Experimental supports evaluation-driven development for AI applications.
21-
22-
[:octicons-arrow-right-24: Core Concepts](core_concepts/index.md)
23-
24-
- 🛠️ **How-to Guides**
25-
26-
Step-by-step guides for specific tasks and goals using Ragas experimental features.
27-
28-
[:octicons-arrow-right-24: How-to Guides](howtos/index.md)
29-
30-
</div>
5+
## Prerequisites
316

7+
- Python 3.9+
8+
- Ragas installed (see [Installation](./install.md))
329

3310
## Hello World 👋
3411

35-
![](hello_world.gif)
12+
![](/_static/imgs/experiments_quickstart/hello_world.gif)
3613

37-
1\. Install Ragas Experimental with local backend
14+
### 1. Install (if you haven’t already)
3815

3916
```bash
40-
pip install ragas-experimental && pip install "ragas-experimental[local]"
17+
pip install ragas
4118
```
4219

43-
2\. Copy this snippet to a file named `hello_world.py` and run `python hello_world.py`
20+
### 2. Create `hello_world.py`
4421

22+
Copy this into a new file and save as `hello_world.py`:
4523

4624
```python
4725
import numpy as np
48-
from ragas_experimental import Dataset
49-
from ragas import experiment
50-
from ragas.metrics import MetricResult, discrete_metric
26+
from ragas import Dataset, experiment
27+
from ragas.metrics import MetricResult, discrete_metric
28+
5129

5230
# Define a custom metric for accuracy
5331
@discrete_metric(name="accuracy_score", allowed_values=["pass", "fail"])
5432
def accuracy_score(response: str, expected: str):
5533
result = "pass" if expected.lower().strip() == response.lower().strip() else "fail"
5634
return MetricResult(value=result, reason=f"Match: {result == 'pass'}")
5735

36+
5837
# Mock application endpoint that simulates an AI application response
5938
def mock_app_endpoint(**kwargs) -> str:
6039
return np.random.choice(["Paris", "4", "Blue Whale", "Einstein", "Python"])
6140

41+
6242
# Create an experiment that uses the mock application endpoint and the accuracy metric
6343
@experiment()
6444
async def run_experiment(row):
6545
response = mock_app_endpoint(query=row.get("query"))
6646
accuracy = accuracy_score.score(response=response, expected=row.get("expected_output"))
6747
return {**row, "response": response, "accuracy": accuracy.value}
6848

49+
6950
if __name__ == "__main__":
7051
import asyncio
71-
52+
7253
# Create dataset inline
7354
dataset = Dataset(name="test_dataset", backend="local/csv", root_dir=".")
7455
test_data = [
@@ -78,22 +59,22 @@ if __name__ == "__main__":
7859
{"query": "Who developed the theory of relativity?", "expected_output": "Einstein"},
7960
{"query": "What programming language is named after a snake?", "expected_output": "Python"},
8061
]
81-
62+
8263
for sample in test_data:
8364
dataset.append(sample)
8465
dataset.save()
85-
66+
8667
# Run experiment
87-
results = asyncio.run(run_experiment.arun(dataset, name="first_experiment"))
68+
_ = asyncio.run(run_experiment.arun(dataset, name="first_experiment"))
8869
```
8970

90-
3\. Check your current directory structure to see the created dataset and experiment results.
71+
### 3. Inspect the generated files
9172

9273
```bash
9374
tree .
9475
```
9576

96-
Output:
77+
You should see:
9778

9879
```
9980
├── datasets
@@ -102,12 +83,18 @@ Output:
10283
└── first_experiment.csv
10384
```
10485

105-
4\. View the results of your first experiment
86+
### 4. View the results of your first experiment
10687

10788
```bash
10889
open experiments/first_experiment.csv
10990
```
11091

111-
Output:
92+
Output preview:
93+
94+
![](/_static/imgs/experiments_quickstart/output_first_experiment.png)
95+
96+
## Next steps
97+
98+
- Learn the concepts behind experiments in [Experiments (Concepts)](../concepts/experimentation.md)
99+
- Explore evaluation metrics in [Metrics](../concepts/metrics/index.md)
112100

113-
![](output_first_experiment.png)

docs/getstarted/index.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# 🚀 Get Started
22

3-
Welcome to the Ragas tutorials! If you're new to Ragas, the Get Started guides will walk you through the fundamentals of working with Ragas. These tutorials assume basic knowledge of Python and building LLM application pipelines.
3+
Welcome to Ragas! If you're new to Ragas, the Get Started guides will walk you through the fundamentals of working with Ragas. These tutorials assume basic knowledge of Python and building LLM application pipelines.
44

55
Before you proceed further, ensure that you have [Ragas installed](./install.md)!
66

@@ -13,4 +13,5 @@ Let's get started!
1313

1414
- [Evaluate your first AI app](./evals.md)
1515
- [Run ragas metrics for evaluating RAG](rag_eval.md)
16-
- [Generate test data for evaluating RAG](rag_testset_generation.md)
16+
- [Generate test data for evaluating RAG](rag_testset_generation.md)
17+
- [Run your first experiment](experiments_quickstart.md)

examples/iterate_prompt/evals.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,7 @@
1010
from run_prompt import run_prompt
1111

1212
from ragas import Dataset, experiment
13-
from ragas.experimental.metrics.discrete import discrete_metric
14-
from ragas.experimental.metrics.result import MetricResult
13+
from ragas.metrics import MetricResult, discrete_metric
1514

1615

1716
@discrete_metric(name="labels_exact_match", allowed_values=["correct", "incorrect"])

examples/ragas_examples/agent_evals/agent.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -303,7 +303,7 @@ def solve(
303303
)
304304

305305
message = response.choices[0].message
306-
messages.append(message.dict())
306+
messages.append(message.model_dump())
307307

308308
self.traces.append(
309309
TraceEvent(

examples/ragas_examples/agent_evals/evals.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
from agent import get_default_agent
1+
from .agent import get_default_agent
22

33
from ragas import Dataset, experiment
44
from ragas.metrics.numeric import numeric_metric

0 commit comments

Comments
 (0)