Ability to filter on modality and dataset size

[shamikbose]

@davanstrien @cakiki
As someone working on this in their spare time, I think it might be helpful to add labels to the datasets indicating their size and modalities. With limited compute, some people might be able to tackle smaller text datasets whereas others with large compute and time might be able to handle large multimodal datasets. I'd be happy to add the labels to the existing datasets if this is something others would like to see as well

In terms of labels, I was thinking of something along these lines

Label	Size
Small	<500MB
Medium	500MB - 2GB
Large	>2GB

[davanstrien]

That's a great idea. We thought of having a difficulty flag for datasets which could also be partially about dataset size.

@cakiki @albertvillanova @clancyoftheoverflow WDYT -- the difficulty rating is probably a little bit harder to determine objectively. We maybe want to include the size info (where the data is in a repository/easily found) so people have a sense of which datasets are easier to work with. We could then encourage people to use the issue itself for more detailed comments about potential challenges i.e. complex data/APIs to work with.

[shamikbose]

I was thinking in terms of building the dataloaders so participiants could filter in the project, but ^ this would be very helpful too!

[davanstrien]

I think this definitely makes a lot of sense (I'm regularly doing the dance of trying to fit datasets onto my laptop which have no business being there!).

[cakiki]

Ah! Apologies for misunderstanding :)

Yes your suggestion makes perfect sense.

[shamikbose]

Pinging @cakiki @davanstrien to see if this can be implemented given that we're adding a lot of new datasets

[davanstrien]

Sorry, I didn't get to this yet, I'll try and get this added to the datasets issue form tomorrow.