-
-
Notifications
You must be signed in to change notification settings - Fork 153
Description
π Feature Description
Add a Python-accessible benchmarking script that measures and compares the
performance of core kornia-imgproc operations against OpenCV equivalents,
with structured output reporting mean time, std deviation, and throughput.
π Feature Category
Performance Optimization
π‘ Motivation
Currently, kornia-rs does not have a reproducible Python-level benchmarking
harness for core imgproc operations. Issue #718 tracks publishing benchmark
results in the README, but there is no tooling for contributors or users to
reproduce those results locally via Python bindings.
This gap becomes critical for the GSoC 2026 ONNX/TensorRT backend project,
where a key deliverable is validating CPU vs CUDA vs TensorRT inference
speedups. Without a baseline imgproc benchmark first, there is no reference
point to measure GPU acceleration gains against.
π Proposed Solution
A benches/python/ directory containing a benchmark script using
pytest-benchmark or timeit that:
- Tests core operations:
resize,warp_affine,rgb_to_grayscale,
gaussian_blur - Runs the same operations using
cv2on identical inputs for comparison - Reports per-operation: mean time (ms), std deviation, and throughput
(images/sec) - Accepts configurable image sizes via CLI args: 256x256, 512x512, 1024x1024
Expected output format:
| Operation | kornia-rs (ms) | OpenCV (ms) | Speedup |
|---|---|---|---|
| resize 512x512 | 0.31 | 0.45 | 1.45x |
| rgb_to_grayscale | 0.12 | 0.18 | 1.50x |
| gaussian_blur | 0.87 | 1.10 | 1.26x |
π Library Reference
- OpenCV Python (
cv2) β used as the performance baseline for all imgproc ops pytest-benchmarkβ https://pytest-benchmark.readthedocs.io/en/latest/- Criterion.rs (existing Rust benchmarks in kornia-rs) β the Python script
mirrors what Criterion already does at the Rust level, but exposes it
through the PyO3 Python bindings layer - NumPy β for generating consistent test input arrays across both libraries
π Alternatives Considered
-
Extending existing Criterion.rs benchmarks: Criterion benchmarks only run
at the Rust level and are not accessible via Python bindings. Users working
through the Python API have no equivalent tool. -
Using
timeitdirectly in a notebook: viable but not reproducible or
CI-friendly. Apytest-benchmarkscript can be integrated into the CI
pipeline and tracked over time.
π― Use Cases
-
GSoC 2026 contributors working on the ONNX/TensorRT backend project need
a baseline benchmark to quantify GPU speedups β this script provides that
reference point. -
Users evaluating kornia-rs vs OpenCV for production pipelines can run
this script locally to make an informed decision based on their hardware. -
Maintainers can integrate it into CI to catch performance regressions
across releases β complementing the benchmark README results in publish benchmark results in README or docsΒ #718.
π Additional Context
This issue is raised as part of pre-application engagement for GSoC 2026,
specifically targeting the "Expand kornia-vlm with ONNX/TensorRT backends"
project. The benchmarking infrastructure proposed here directly supports
that project's deliverable of comparing CPU vs CUDA vs TensorRT inference.
I am happy to implement this as my pre-application contribution if a
maintainer confirms the scope and preferred tooling (pytest-benchmark
vs timeit vs custom harness).
Related: #718 (publish benchmark results in README/docs)
π€ Contribution Intent
- I plan to submit a PR to implement this feature
- I'm requesting this feature but not planning to implement it