[BUG] Hyperparameter tuning for next item recommenders

[logrkn]

I'm trying to do hyperparameter tuning for a next item recommender using yoochoose data (code below) but get the following error:

ValueError: model must be a NextItemRecommender but '<class 'cornac.hyperopt.RandomSearch'>' is provided

What is the correct approach for tuning next item recommenders?

import cornac
from cornac.data import Reader
from cornac.datasets import yoochoose
from cornac.eval_methods import NextItemEvaluation
from cornac.metrics import NDCG
from cornac.models import GRU4Rec
from cornac.hyperopt import Discrete, Continuous
from cornac.hyperopt import RandomSearch

buy_data = yoochoose.load_buy(reader=Reader(min_sequence_size=2, num_top_freq_item=30000))

item_set = set([tup[1] for tup in buy_data])

test_data = yoochoose.load_test(reader=Reader(min_sequence_size=2, item_set=item_set))

next_item_eval = NextItemEvaluation.from_splits(
    train_data=buy_data[:10000],
    test_data=test_data[:10000],  
    exclude_unknowns=True,
    verbose=False,
    fmt="SITJson",
)

ndcg20 = NDCG(20)

gr = GRU4Rec(layers=64, loss="bpr-max", n_sample=2048, n_epochs=5, seed=123, verbose=False)

rs_gr = RandomSearch(model=gr, 
                      space=[Discrete(name="layers", values=[64, 128, 256])], 
                      metric=ndcg20,
                      eval_method=next_item_eval,
                      n_trails=10)

cornac.Experiment(eval_method=next_item_eval, 
                  models=[rs_gr], 
                  metrics=[ndcg20], 
                  show_validation=False).run()

[qtuantruong]

@lthoang is it something that we can try to address?

[lthoang]

@tqtg Up to the current release 2.2.2, we haven't support hyperparameter tuning for NextItemRecommender model. We need another PR to support this feature.

@logrkn You can apply the following snippet to iterate through your candidate list of parameters:

import cornac
from cornac.data import Reader
from cornac.datasets import yoochoose
from cornac.eval_methods import NextItemEvaluation
from cornac.metrics import NDCG
from cornac.models import GRU4Rec
from cornac.hyperopt import Discrete, Continuous
from cornac.hyperopt import RandomSearch

buy_data = yoochoose.load_buy(reader=Reader(min_sequence_size=2, num_top_freq_item=30000))

item_set = set([tup[1] for tup in buy_data])

test_data = yoochoose.load_test(reader=Reader(min_sequence_size=2, item_set=item_set))

next_item_eval = NextItemEvaluation.from_splits(
    train_data=buy_data[:10000],
    test_data=test_data[:10000],  
    exclude_unknowns=True,
    verbose=False,
    fmt="SITJson",
)

models = [
    GRU4Rec(
        name="GRU4Rec_layer_{}".format(layer),
        layers=[layer], 
        loss="bpr-max",
        n_sample=2048,
        n_epochs=5,
        seed=123,
        verbose=False
    )
    for layer in [64, 128, 256]
]

cornac.Experiment(eval_method=next_item_eval, 
                  models=models, 
                  metrics=[NDCG(20)], 
                  show_validation=False).run()

[logrkn]

Thank you, I'll try that. I would be great to get hyperparameter tuning included for NextItemRecommender models in the future though as ideally I'd like to test more combinations than in this simple example.