[Feature]: Add execution provider selection and inference benchmarking

[fmutlu68]

🚀 Feature Description

Add execution provider selection (CPU/CUDA) and multi-iteration inference benchmarking to the existing examples/onnx example.

📂 Feature Category

Testing Infrastructure

💡 Motivation

The current examples/onnx example only runs inference using the default CPU execution provider with a single pass. This doesn't reflect real-world applications where backend selection and accurate latency measurement are critical. When evaluating models for deployment on different hardware (CPU vs GPU), having a quick way to compare execution providers within the same example is essential.

💭 Proposed Solution

• Add a --device CLI flag to select the execution provider (cpu or cuda)
• Add a --num-iterations CLI flag for multi-run benchmarking (default: 10)
• Add a warm-up run before timed iterations to exclude cold-start overhead
  (memory allocation, graph optimization, kernel compilation)
• Print latency statistics (mean, min, max) across iterations
• Update README with usage examples for both CPU and CUDA execution

📚 Library Reference

• ONNX Runtime Execution Providers: https://onnxruntime.ai/docs/execution-providers/
• ort crate CUDA EP: https://docs.rs/ort/latest/ort/execution_providers/cuda/
• Current examples/onnx implementation in this repo

🔄 Alternatives Considered

Considered creating a completely new example, but improving the existing one keeps codebase lean and provides immediate value to current users of the ONNX example.

🎯 Use Cases

• Comparing inference latency across CPU and CUDA backends for model deployment decisions
• Providing a starting point for users who want to run ONNX models with GPU acceleration
• Laying groundwork for TensorRT execution provider support (related: [Feature]: Add ONNXRuntime-based Vision-Language Model (VLM) inference example to kornia-vlm #634,
  GSoC 2026 VLM Inference project)

📝 Additional Context

This is a small, focused improvement to an existing example. The changes are compatible, default behavior remains CPU with a single run if no extra flags are provided.

🤝 Contribution Intent

• I plan to submit a PR to implement this feature
• I'm requesting this feature but not planning to implement it

[github-actions]

⚠️ Before Submitting a PR

Please wait for a maintainer to approve and assign this issue to you or someone else before submitting a PR. PRs submitted without prior approval and assignment may be closed. See our Contributing Guide for details.

[fmutlu68]

@sidd-27 @edgarriba @cjpurackal could assign me this issue, i want to work on it

[Aditya-ha11]

Hi, I’m interested in working on this issue as part of my GSoC 2026 preparation.

I have experience in computer vision and deep learning (PyTorch-based pipelines), and I’m currently working on efficient vision-language model (VLM) inference. This issue aligns closely with my proposed work on optimizing VLM inference pipelines using ONNX Runtime and TensorRT.

From the description, I understand the goal is to extend the existing ONNX example by:

• Adding a --device flag for CPU/CUDA selection
• Supporting multi-iteration benchmarking with warm-up runs
• Reporting latency statistics (mean/min/max)

I briefly looked into the examples/onnx implementation and noticed it currently defaults to CPU execution with a single run, so extending it for configurable providers and benchmarking would be very useful.

I will explore how execution providers are currently configured in the codebase and extend it with configurable providers and benchmarking utilities.

Could you please guide me on:

1. Where execution providers are currently initialized in the codebase?
2. Any preferred benchmarking utilities or patterns used in this repo?

I’d be happy to start with an initial implementation once I get your direction.