Tutorials for other datasets?

[slerman12]

Are there any tutorials for datasets besides Imagenet? Like TinyImagenet or Cifar-100 for example?

[rwightman]

@slerman12 not at this time, the best resource right now with a few demos is the colab notebook I recently shared https://twitter.com/wightmanr/status/1459314881534971904 (https://colab.research.google.com/drive/1czzQ8sUe-6rCxlK4vRCUk230pJuCSSS4?usp=sharing) cifar100 usage is shown there

You can see the supported torchvision datasets in the factory https://github.com/rwightman/pytorch-image-models/blob/master/timm/data/dataset_factory.py ... each torchvision dataset has a slightly different API for creation so I can't just blanket cover them all.

TFDS wrapper doesn't have any per-dataset limitation, you can try all of them, but they must be image classification datasets (or have a 'image' and 'label' field in the example encoding to work without modifications, so most of these should work https://www.tensorflow.org/datasets/catalog/overview#image_classification

For the train script you need to specify the dataset source and name on the command line with --dataset

ie --dataset tfds/caltech_birds2011 ... if the name of the validation split for the dataset differs from the default validation or val you need to specify it manually, ie for birds there is no validation split, only train and test, so you might want to use --val-split test if you don't care to maintain val and test, or you could use the TFDS splitting feature and use --val-split test[:50%] to use half of test split for val and then you can test on test[50%:]

I'm not too familiar with the tiny imagenet form, I don't think it's in torchvision or TFDS but I'm pretty sure it's just a folder dataset (folder per class) so you should be able to just set the dataset root dir to the location as long as train and val/eval/validation/valid folders exist.

Make sure you set --num-classes arg on cmd line to match classes of your dataset

[ireneb612]

Do you know if the --val-split test[:50%] can also be used on custom data?

[rwightman]

@ireneb612 yes, those splits work for any TFDS based dataset and is consitent across all uses ... BUT it's a bit of a pain to create custom TFDS datasets (but probably worth it if you are going to do extensive experiments with a given dataset, especially in cloud)

See: https://www.tensorflow.org/datasets/add_dataset

[charchit7]

I @rwightman I loved the library, I was wondering if you know any repo that created a tutorial on CIFAR100 that would help. It's been a year so maybe :) thanks for creating timm