Skip to content

Problem with evaluating "termfreq" model #41

@harkous

Description

@harkous

Hi,

Thanks for this useful project.
There is an issue I guess in the evaluation of the termfreq model.

I'm running:
python3 tools/train.py termfreq anssel ./data/anssel/wang/train.csv ./data/anssel/wang/test.csv inp_e_dropout=1/2 nb_epoch=1

However, I get this error:

"... tools/pysts/eval.py", line 28, in binclass_accuracy rawacc = np.sum((ypred > 0.5) == (y > 0.5)) / ypred.shape[0]
TypeError: unorderable types: dict() > float()

The origin of this issue is that y_pred should be replaced by y_pred['score'] for this particular task (as the function predict in termfreq.py returns a dictionary).

This is also the case in the other function aggregate_s0.

Still after fixing this, I get another issue:

... /tools/pysts/eval.py", line 122, in mrr if yy[1] in ysd: TypeError: unhashable type: 'numpy.ndarray'

I appreciate your feedback on this issue or on whether I am running something incorrectly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions