Skip to content

Conversation

@grgkovac
Copy link

@grgkovac grgkovac commented Jan 21, 2026

Summary

This PR addresses #1638 by improving the error handling when a classification metric cannot find a specified (or default) classification ID in the DataDefinition.

The Problem

Currently, if a user defines a BinaryClassification with a custom name, but forgets to specify that classification_name in their Metric, the system returns None. This results in a cryptic KeyError: None deep in the library.

See issue #1638

Code example:

import pandas as pd
from evidently import Dataset, DataDefinition, Report, BinaryClassification
from evidently.metrics import Accuracy

data = pd.DataFrame({"id_col": [1, 2, 3], "actual": [0, 1, 1], "predicted": [1, 0, 1]})
classification_ = [BinaryClassification(target="actual", prediction_labels="predicted", name="cls")]
data_definition = DataDefinition(id_column="id_col", classification=classification_)
dataset = Dataset.from_pandas(data, data_definition=data_definition)
report = Report(metrics=[Accuracy(target="actual", prediction="predicted", name="acc")]) # <- missing: classification_name="cls"
report.run(dataset)

The Fix

I modified get_classification in datasets.py to raise a descriptive ValueError instead of returning None. This follows the "fail fast" principle and provides the user with actionable feedback.

New Error Message:
ValueError: No classification with id 'default' found.

I can potentially add something like this as well?
If you are using custom names, ensure you pass 'classification_name' to your Metric (e.g., Accuracy(..., classification_name='your_name')).


Fixes #1638

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BinaryClassification with name != "default" crashes

1 participant