Skip to content

Commit 19ecd56

Browse files
authored
docs: improve hello world (#2127)
1 parent 5ac7608 commit 19ecd56

File tree

8 files changed

+36
-29
lines changed

8 files changed

+36
-29
lines changed
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

docs/experimental/hello_world.gif

1.71 MB
Loading

docs/experimental/index.md

Lines changed: 34 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,61 +1,58 @@
11
# Ragas Experimental
22

3+
A framework for applying Evaluation-Driven Development (EDD) to AI applications.
4+
5+
The goal of Ragas Experimental is to evolve Ragas into a general-purpose evaluation framework for AI applications. It helps teams design, run, and reason about evaluations across any AI workflow. Beyond tooling, it provides a mental model for thinking about evaluations not just as a diagnostic tool, but as the backbone of iterative improvement.
6+
37
# ✨ Introduction
48

59

610
<div class="grid cards" markdown>
11+
712
- 🚀 **Tutorials**
813

9-
Install with `pip` and get started with Ragas with these tutorials.
14+
Step-by-step guides to help you get started with Ragas Experimental. Learn how to evaluate AI applications like RAGs and agents with practical examples.
1015

1116
[:octicons-arrow-right-24: Tutorials](tutorials/index.md)
1217

13-
- 📚 **Core Concepts**
18+
- 📚 **Explanation**
1419

15-
In depth explanation and discussion of the concepts and working of different features available in Ragas.
16-
17-
[:octicons-arrow-right-24: Core Concepts](core_concepts/index.md)
20+
A deeper dive into the principles of evaluation and how Ragas Experimental supports evaluation-driven development for AI applications.
1821

22+
[:octicons-arrow-right-24: Explanation](explanation/index.md)
1923

2024
</div>
2125

22-
## Installation
23-
24-
- Install ragas_experimental from pip
2526

26-
```bash
27-
pip install ragas_experimental
28-
```
27+
## Hello World 👋
2928

30-
- Install from source
29+
![](hello_world.gif)
3130

32-
```bash
33-
git clone https://github.com/explodinggradients/ragas
34-
```
31+
1\. Install Ragas Experimental with local backend
3532

3633
```bash
37-
cd ragas/experimental && pip install -e .
34+
pip install ragas-experimental && pip install "ragas-experimental[local]"
3835
```
3936

37+
2\. Copy this snippet to a file named `hello_world.py` and run `python hello_world.py`
4038

41-
## Hello World 👋
42-
43-
Copy this snippet to a file named `hello_world.py` and run `python hello_world.py`
4439

4540
```python
4641
import numpy as np
4742
from ragas_experimental import experiment, Dataset
48-
from ragas_experimental.metrics import MetricResult, numeric_metric
49-
43+
from ragas_experimental.metrics import MetricResult, discrete_metric
5044

51-
@numeric_metric(name="accuracy_score", allowed_values=(0, 1))
45+
# Define a custom metric for accuracy
46+
@discrete_metric(name="accuracy_score", allowed_values=["pass", "fail"])
5247
def accuracy_score(response: str, expected: str):
53-
result = 1 if expected.lower().strip() == response.lower().strip() else 0
54-
return MetricResult(result=result, reason=f"Match: {result == 1}")
48+
result = "pass" if expected.lower().strip() == response.lower().strip() else "fail"
49+
return MetricResult(value=result, reason=f"Match: {result == 'pass'}")
5550

51+
# Mock application endpoint that simulates an AI application response
5652
def mock_app_endpoint(**kwargs) -> str:
5753
return np.random.choice(["Paris", "4", "Blue Whale", "Einstein", "Python"])
5854

55+
# Create an experiment that uses the mock application endpoint and the accuracy metric
5956
@experiment()
6057
async def run_experiment(row):
6158
response = mock_app_endpoint(query=row.get("query"))
@@ -83,7 +80,13 @@ if __name__ == "__main__":
8380
results = asyncio.run(run_experiment.arun(dataset, name="first_experiment"))
8481
```
8582

86-
View Results
83+
3\. Check your current directory structure to see the created dataset and experiment results.
84+
85+
```bash
86+
tree .
87+
```
88+
89+
Output:
8790

8891
```
8992
├── datasets
@@ -92,8 +95,12 @@ View Results
9295
└── first_experiment.csv
9396
```
9497

95-
Open the results in a CSV file
98+
4\. View the results of your first experiment
9699

97100
```bash
98101
open experiments/first_experiment.csv
99-
```
102+
```
103+
104+
Output:
105+
106+
![](output_first_experiment.png)
149 KB
Loading

experimental/ragas_experimental/backends/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,14 +38,14 @@ class MyBackend(BaseBackend):
3838
```toml
3939
# pyproject.toml
4040
[project.entry-points."ragas.backends"]
41-
"my_storage" = "my_package.backend:MyBackend"
41+
"my_backend" = "my_package.backend:MyBackend"
4242
```
4343

4444
**3. Use:**
4545
```python
4646
from ragas_experimental.backends import get_registry
4747
registry = get_registry()
48-
backend = registry["my_storage"](connection_string="...")
48+
backend = registry["my_backend"](connection_string="...")
4949
```
5050

5151
## Required Methods

0 commit comments

Comments
 (0)