Requesting a "difficulty" column in the dataset characteristic table

[ziadloo]

First of all, thanks for the datasets. Secondly, this is a feature request/suggestion.

I think it would help a lot if this dataset collection came with a difficulty measure for each dataset. The candidate that comes to my mind is the best performance provided by any solution so far. Or the average and standard deviation of all the solutions.

I understand the problems with such measures (for one, how to make a fair judgment) but at any rate, I just wanted to say that as a user, I'm having a hard time choosing which dataset to use. Or knowing if the performance of my solution is any good.

Thanks

[gkronber]

I like the idea, because several datasets could be marked as easy (expressions such as x*y or x/y from the Feynman problem instances come to mind).

It will however be difficult to come up with an objective measure of difficulty. As it depends on several factors:

• Size and dimensionality of the dataset
• Nonlinearity
• Complexity of the model required to predict the target well.
• Noise

If your main motivation is "knowing if the performance of my solution is any good" maybe it would be better to work with SRBench which is specifically focused on comparison to other SR implementations.