How to find epochs for best model when using tuner.search()

[a2smith]

We can look at the code here for this question: https://keras.io/guides/keras_tuner/getting_started/

I have defined a build_model function similar to the example. My tuner definition is similar as well except I'm using BayesOptimization.

I then do search with max epochs = 900 and have a stop_early callback

stop_early = EarlyStopping(monitor='val_loss', min_delta=0.0001, patience=30, restore_best_weights=True) 
tuner.search(X_train, y_train, epochs=900, validation_split=0.11, callbacks=[stop_early])

That works fine and a best model is found. Then I get the 10 best models and save the best model in best_model.

models = tuner.get_best_models(num_models=10)
best_model = models[0]

Now, I'm trying to retrain the model with all of the training data as suggested. The example shows epochs=1 when retraining.

model.fit(x=x_all, y=y_all, epochs=1) #in the example, this isn't my code

I want to train for the same number of epochs that the stop_early callback found when doing the optimization. I haven't found a way to look up the best_step value. I even tried manually checking the trial_nnn folders in the project folder but the trial.json file shows "best_step": 0, for ALL of the trials. It seems like the best_step value isn't being saved.

How can I find out the number of epochs in each trial that the stop_early callback found the best weights? I want to set epochs equal to that value when I call model.fit().

[haifeng-jin]

Sorry, I do not see a clear way to do it.
You may need to hack the source code where it copies the callbacks.
https://github.com/keras-team/keras-tuner/blob/master/keras_tuner/engine/tuner.py#L288

[a2smith]

Thanks for the fast reply. I found a "brute force" workaround. I build model with best hyperparameters and then do model fit with same validation split and early stopping criteria. Then, I find minimum val_loss in model fit history and look up its list index. Finally, I build model again to reset weights, etc and then do fit with all training data for the best number of epochs so I don't overfit.

# Build the model with the best hp.
model = build_model(best_hps)

# First, fit again with same validation split to find out best # of epochs
stop_early = EarlyStopping(monitor='val_loss', min_delta=0.0001, patience=30, restore_best_weights=True)
history1 = model.fit(X_train, y_train, epochs=900, validation_split=0.11, callbacks=[stop_early])
best_epoch = history1.history['val_loss'].index(min(history1.history['val_loss']))

# Then, do model fit again with all data and same number of epochs
model = build_model(best_hps)
history2 = model.fit(X_train, y_train, epochs=best_epoch)