Upgrade to 0.10.0: "old" EasyTrain does not generate "validate" metric anymore

[hmf]

Hi,

I have only now realized that the "validate" timing metric is not created after the upgrade top 0.10. The log file also does not appear. I am using an old version of EasyTrain to perform the training (version 0.9.0) and validation. Looking at the code, the only change seems to be the support for multi-threaded training.

Do I have to do some additional set-up to get the validation timings also?

Note: I am using a slightly changed version of EasyTrain because I am using the labeled MNIST dataset, but because I am testing an autoencoder, I need to predict and compare the image itself.

TIA

[frankfliu]

Are you using EasyTrain.fit()?
The validation metric was generated in: https://github.com/deepjavalibrary/djl/blob/master/api/src/main/java/ai/djl/training/EasyTrain.java#L70

            evaluateDataset(trainer, validateDataset);

I verified the master branch, we are still generating "validate" metric in TrainMNIST example.

[hmf]

@frankfliu

Are you using EasyTrain.fit()?

I am using a slightly altered version of DJL 0.9.0

The validation metric was generated in: https://github.com/deepjavalibrary/djl/blob/master/api/src/main/java/ai/djl/training/EasyTrain.java#L70

I am assuming the actual metric is added here:

    NDList preds = trainer.evaluate(data);

so yes, I am adding it.

I have confirmed this is working. The problem was (is?) that I loaded the MNIST test set using a RandomAccessDataset and set the optLimit to 1000. The result was that the ProgressBar appeared only once (first batch) and the metric was not set.

I assume the metric should be added irrespective of the limit, i.e.: for every batch at leas one value of the validation metric should be added. If this is not so, please let me know.

Thanks for the help.

[frankfliu]

Trainer.evaluate() itself wont' create metric entry, the metric entry is created in

trainer.notifyListeners(listener -> listener.onValidationBatch(trainer, batchData));

[hmf]

Ok. It is called, so no problem here. Must be some condition that is not satisfied when I limit the number of samples. Regardless, when I use all the data it works.

Thank you.