You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
from evidently.future.descriptors import Sentiment, TextLength, Contains
96
96
from evidently.future.report import Report
97
-
from evidently.future.presets import DataDriftPreset
97
+
from evidently.future.presets import TextEvals
98
+
```
98
99
99
-
iris_data = datasets.load_iris(as_frame=True)
100
-
iris_frame = iris_data.frame
100
+
Create a toy dataset with questions and answers.
101
+
102
+
```python
103
+
eval_df = pd.DataFrame([
104
+
["What is the capital of Japan?", "The capital of Japan is Tokyo."],
105
+
["Who painted the Mona Lisa?", "Leonardo da Vinci."],
106
+
["Can you write an essay?", "I'm sorry, but I can't assist with homework."]],
107
+
columns=["question", "answer"])
101
108
```
102
109
103
-
Run the **Data Drift** evaluation preset that will test for shift in column distributions. Take the first 60 rows of the dataframe as "current" data and the following as reference. Get the output in Jupyter notebook:
110
+
Create an Evidently Dataset object and add `descriptors`: row-level evaluators. We'll check for sentiment of each response, its length and whether it contains words indicative of denial.
You can also save an HTML file. You'll need to open it from the destination folder.
122
+
You can view the dataframe with added scores:
115
123
116
124
```python
117
-
my_eval.save_html("file.html")
125
+
eval_dataset.as_dataframe()
118
126
```
119
127
120
-
To get the output as JSON or Python dictionary:
128
+
To get a summary Report to see the distribution of scores:
129
+
121
130
```python
122
-
my_eval.json()
131
+
report = Report([
132
+
TextEvals()
133
+
])
134
+
135
+
my_eval = report.run(eval_dataset)
136
+
my_eval
137
+
# my_eval.json()
123
138
# my_eval.dict()
124
139
```
125
-
You can choose other Presets, create Reports from indiviudal Metrics and configure pass/fail conditions.
140
+
You can also choose other evaluators, including LLM-as-a-judge and configure pass/fail conditions.
126
141
127
142
### Data and ML evals
128
143
@@ -166,7 +181,8 @@ my_eval.json()
166
181
You can choose other Presets, create Reports from indiviudal Metrics and configure pass/fail conditions.
167
182
168
183
## Monitoring dashboard
169
-
> This launches a demo project in the Evidently UI. Check tutorials for [Self-hosting](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-monitoring) or [Evidently Cloud](https://docs.evidentlyai.com/tutorials-and-examples/tutorial-cloud).
184
+
185
+
> This launches a demo project in the locally hosted Evidently UI. Sign up for [Evidently Cloud](https://docs.evidentlyai.com/docs/setup/cloud) to instantly get a managed version with additional features..
170
186
171
187
Recommended step: create a virtual environment and activate it.
0 commit comments