validation loss on eval_every_steps ?

[leo90v]

Hi,

I'm using the eval_every_steps parameter for early stopping. I noticed it also stocks the metrics on the train_val.zlog file but would it be possible to also stock the loss at that evaluation like it is done for the best model on the val_metrics.json file ?

I would like to compare my training and evaluation losses if possible.

Thanks !

[sleepinyourhat]

I'm not sure— @zphang, ming taking a look?

[zphang]

Hi,

This function call is where the the metrics are extracted for each of the val_result dictionaries.

jiant/jiant/proj/main/metarunner.py

Lines 231 to 233 in 235f646

	val_metrics_dict = jiant_task_sampler.get_metrics_dict_from_results_dict(
	results_dict=val_results_dict,
	)

However, the dictionary should also contain a key for the loss value. So you can try modifying code are that segment to save the loss as well.