You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Evidently is an open-source Python library for ML and LLM evaluation and observability. It helps evaluate, test, and monitor AI-powered systems and data pipelines from experimentation to production.
32
28
33
-
* 🔡 Works with tabular, text data, and embeddings.
29
+
* 🔡 Works with tabular and text data.
34
30
* ✨ Supports predictive and generative systems, from classification to RAG.
35
31
* 📚 100+ built-in metrics from data drift detection to LLM judges.
36
-
* 🛠️ Python interface for custom metrics and tests.
32
+
* 🛠️ Python interface for custom metrics.
37
33
* 🚦 Both offline evals and live monitoring.
38
34
* 💻 Open architecture: easily export data and integrate with existing tools.
39
35
40
-
Evidently is very modular. You can start with one-off evaluations using `Reports`or `Test Suites`in Python or get a real-time monitoring `Dashboard` service.
36
+
Evidently is very modular. You can start with one-off evaluations using `Reports` in Python or host a monitoring `Dashboard` service.
41
37
42
-
## 1. Reports
38
+
## 1. Reports and Test Suites
43
39
44
-
**Reports** compute various data, ML and LLM quality metrics. You can start with Presets or customize.
40
+
**Reports** compute and summarize various data, ML and LLM quality metrics. You can start with Presets or customize.
45
41
* Out-of-the-box interactive visuals.
46
-
* Best for exploratory analysis and debugging.
47
-
* Get results in Python, export as JSON, Python dictionary, HTML, DataFrame, or view in monitoring UI.
> This is a simple Hello World. Check the Tutorials for more: [Tabular data](https://docs.evidentlyai.com/tutorials-and-examples/tutorial_reports_tests) or [LLM evaluation](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-llm).
83
+
## Reports
84
+
85
+
### LLM evals
86
+
87
+
> This is a simple Hello World. Check the Tutorials for more: [Tabular data](https://docs.evidentlyai.com/quickstart_ml) or [LLM evaluation](https://docs.evidentlyai.com/quickstart_llm).
96
88
97
89
Import the **Test Suite**, evaluation Preset and toy tabular dataset.
98
90
@@ -130,7 +122,7 @@ data_stability.json()
130
122
```
131
123
You can choose other Presets, individual Tests and set conditions.
132
124
133
-
### Option 2: Reports
125
+
### Data and ML evals
134
126
135
127
Import the **Report**, evaluation Preset and toy tabular dataset.
136
128
@@ -168,7 +160,7 @@ data_drift_report.json()
168
160
169
161
You can choose other Presets and individual Metrics, including LLM evaluations for text data.
170
162
171
-
### Option 3: ML monitoring dashboard
163
+
##Monitoring dashboard
172
164
> This launches a demo project in the Evidently UI. Check tutorials for [Self-hosting](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-monitoring) or [Evidently Cloud](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-cloud).
173
165
174
166
Recommended step: create a virtual environment and activate it.
@@ -187,7 +179,7 @@ Access Evidently UI service in your browser. Go to the **localhost:8000**.
187
179
188
180
# 🚦 What can you evaluate?
189
181
190
-
Evidently has 100+ built-in evals. You can also add custom ones. Each metric has an optional visualization: you can use it in `Reports`, `Test Suites`, or plot on a `Dashboard`.
182
+
Evidently has 100+ built-in evals. You can also add custom ones.
191
183
192
184
Here are examples of things you can check:
193
185
@@ -207,16 +199,7 @@ Here are examples of things you can check:
207
199
We welcome contributions! Read the [Guide](CONTRIBUTING.md) to learn more.
208
200
209
201
# :books: Documentation
210
-
For more information, refer to a complete <ahref="https://docs.evidentlyai.com">Documentation</a>. You can start with the tutorials:
211
-
*[Get Started with Tabular and ML Evaluation](https://docs.evidentlyai.com/tutorials-and-examples/tutorial_reports_tests)
212
-
*[Get Started with LLM Evaluation](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-llm)
213
-
*[Self-hosting ML monitoring Dashboard](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-monitoring)
214
-
*[Cloud ML monitoring Dashboard](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-cloud)
215
-
216
-
See more examples in the [Docs]([https://docs.evidentlyai.com/tutorials-and-examples](https://docs.evidentlyai.com/tutorials-and-examples/examples)).
217
-
218
-
## How-to guides
219
-
Explore the [How-to guides](https://github.com/evidentlyai/evidently/tree/main/examples/how_to_questions) to understand specific features in Evidently.
202
+
For more examples, refer to a complete <ahref="https://docs.evidentlyai.com">Documentation</a>.
220
203
221
204
# :white_check_mark: Discord Community
222
205
If you want to chat and connect, join our [Discord community](https://discord.gg/xZjKRaNp8b)!
0 commit comments