Model test on "train split" in DTD dataset ?

[xyzz-jpg]

Dear authors,

Thank you for your significant contribution to the community.

While debugging the code, I discovered a potential issue. It appears that the model is being tested on the training set for the DTD dataset. Following the code you released, I was able to reproduce the results in the paper under the real finetune setting, using only real data. However, when I modified the code to test on the test set and followed the hyperparameter search strategy described in the paper, I could only achieve a result of 74.5.

I look forward to your response.

Thank you very much!

DataDream/classify/data.py

Line 673 in a71515f

test_dataset = split_dtd(real_train_data_dir, test_transform, 'train')