Skip to content

[BUG] Experiment fails without warning if list of models is empty #674

@miskfi

Description

@miskfi

Description

If the list of models in cornac.Experiment after validation ends up being empty, Experiment.run() fails with IndexError: list index out of range in cornac/experiment/.result.py:190 when trying to print the test results, because self.result is an empty list.

It would be nice if Experiment.run() reported that the list of loaded models is empty. The current error message is quite unclear and I had to debug what exactly was causing the error.

In which platform does it happen?

Any platform, cornac==2.3.5

How do we replicate the issue?

Running

import random

import cornac
from cornac.eval_methods import NextBasketEvaluation
from cornac.metrics import NDCG, Recall
from cornac.models import UPCF

dummy_data = [tuple(random.randint(0, 100) for _ in range(3)) for _ in range(50)]
next_basket_eval = NextBasketEvaluation(
    data=dummy_data, fmt="UBI", seed=123, verbose=True
)

# I accidentally added a comma that made it into a ([UPCF()],) tuple -> this results in an empty self.models list in cornac.Experiment
models = [UPCF()],  

metrics = [Recall(k=10), Recall(k=20), NDCG(k=10), NDCG(k=20)]

cornac.Experiment(eval_method=next_basket_eval, models=models, metrics=metrics).run()

gives IndexError, which makes it unclear that the issue was that the list of models was empty.

Expected behavior (i.e. solution)

It would be nice to add an error message that would directly warn that the list of loaded models is empty.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions