How to use it?

[pplonski]

How can I use this benchmark suite? I'm working on AutoML https://github.com/mljar/mljar-supervised and would love to have a quick way to run tests.

Does it integrate with GitHub Actions?

[cweill]

Note: We're still in the pre-release stage, so caveat emptor. :)

@pplonski: Thanks for reaching out. If you want to try running our benchmark suite, follow the instructions at https://github.com/google/nitroml/tree/master/examples. This suite is designed to be run on GCP using KubeFlow.

Integration is straightforward if you are already using TensorFlow, i.e. accepting TensorFlow Examples as inputs, and exporting a TensorFlow SavedModel. Does MLJar support that? If so I can point you in the right direction. Otherwise, we are planning to make this this framework more ML framework agnostic in Q4 2020, i.e. accepting raw CSV data as inputs, and exporting numpy predictions.

[cweill]

@pplonski: You mentioned integrating with GitHub actions which sounds like a good idea: could you explain what you're envisioning?