NewFeatureIdeas

Ideas

New features:

1. Extend the set of inference frameworks and models in regular benchmarks: TensorRT (Python API), ExecuTorch (Python API), JAX/Flax (for Google TPUs).
2. Study of methods for predicting inference performance (using machine learning).
3. Extend the set of hardware platforms for regular performance measurements (RISC-V, RaspberryPi 4 8GB) (if we will have access to hardware).
4. Update GUI application to create configuration files for benchmarking.
5. PyTorch quantization support.
6. ONNX Runtime quantization support.
7. Intel Extension for PyTorch (for iGPU) support.
8. Enabling collection of layer-by-layer performance statistics for models. − Study the internal capabilities of frameworks (OpenVINO – Benchmark Tool, TVM – [Performance Application Programming Interface (PAPI)][https://tvm.apache.org/docs/how_to/profile/papi.html], PyTorch, ONNX Runtime, TensorFlow, TensorFlow Lite). − Provide support for the main frameworks whose output is supported in the benchmarking system.
9. Implement Python wrappers for the C++ version of the benchmark (for example, using pybind11).
10. An application (first a script) for visualizing the results of inference performance. − Input data: models, a set of frameworks, a set of batch sizes, other parameters. − Output data: inference performance graphs built using the matplotlib Python-package.