Computing metafeatures without uploading the dataset

[montali]

Hi OpenML! I'm currently doing some research on a possible integration of a meta-model built on OpenML data into AutoML pipelines. I tried looking around for the actual code that computes the data qualities, but there seem to be multiple libraries under the organization.
I was wondering: what's the current way you use to compute these metafeatures? Is it possible to extract this, or do I have to go through the website (possibly waiting for the computation to happen)?
Thanks!

[joaquinvanschoren]

You can compute all data qualities using any of the APIs, e.g. using Python: https://openml.github.io/openml-python/main/api.html#dataset-functions If you want to compute them yourself, the current implementation is here: https://github.com/openml/EvaluationEngine However, we are moving to a new library that should, in principle, be able to compute the same values: https://github.com/ealcobaca/pymfe Hope that helps, Joaquin

…

On Fri, 29 Jul 2022 at 09:21, Simone Montali ***@***.***> wrote: Hi OpenML! I'm currently doing some research on a possible integration of a meta-model built on OpenML data into AutoML pipelines. I tried looking around for the actual code that computes the data qualities, but there seem to be multiple libraries under the organization. I was wondering: what's the current way you use to compute these metafeatures? Is it possible to extract this, or do I have to go through the website (possibly waiting for the computation to happen)? Thanks! — Reply to this email directly, view it on GitHub <#1160>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANFAVYALJD3EMHJTETZWEDVWOA7FANCNFSM547YQGHA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[montali]

Hey Joaquin! Thanks for the great answer. From what I see, I can use the API and run create_dataset, but that requires uploading the dataset to the service. EvaluationEngine, on the other hand, seems to be what I'm looking for. Is there any way to use it as a CLI providing the dataset, instead of having it connect to the API?
Thanks a lot!

[joaquinvanschoren]

Hi Simone,

If you want the qualities of a dataset already existing on OpenML, you can simply use get_dataset(id) and then data.qualities.
If you want to compute the qualities on a new dataset, I would recommend using pymfe.
See eg. the example here: https://pymfe.readthedocs.io/en/latest/auto_examples/01_introductory_examples/plot_pymfe_default.html#sphx-glr-auto-examples-01-introductory-examples-plot-pymfe-default-py

The evaluation engine has a CLI, but it's designed to take an OpenML dataset ID and uploads the qualities directly to the platform. It will be a lot of work to adapt it.

[joaquinvanschoren]

Sidenote: we are moving towards adopting pymfe in the new version of the evaluation engine as well.

[montali]

I was looking for parity with OpenML data, as the meta-model has been trained on that; I'll try to get to parity with pyMFE and keep you posted! Thanks a lot again!