Skip to content

[Feature]: Add Python benchmarking script to compare kornia-rs vs OpenCV for core imgproc operationsΒ #784

@Kiran95021

Description

@Kiran95021

πŸš€ Feature Description

Add a Python-accessible benchmarking script that measures and compares the
performance of core kornia-imgproc operations against OpenCV equivalents,
with structured output reporting mean time, std deviation, and throughput.

πŸ“‚ Feature Category

Performance Optimization

πŸ’‘ Motivation

Currently, kornia-rs does not have a reproducible Python-level benchmarking
harness for core imgproc operations. Issue #718 tracks publishing benchmark
results in the README, but there is no tooling for contributors or users to
reproduce those results locally via Python bindings.

This gap becomes critical for the GSoC 2026 ONNX/TensorRT backend project,
where a key deliverable is validating CPU vs CUDA vs TensorRT inference
speedups. Without a baseline imgproc benchmark first, there is no reference
point to measure GPU acceleration gains against.

πŸ’­ Proposed Solution

A benches/python/ directory containing a benchmark script using
pytest-benchmark or timeit that:

  1. Tests core operations: resize, warp_affine, rgb_to_grayscale,
    gaussian_blur
  2. Runs the same operations using cv2 on identical inputs for comparison
  3. Reports per-operation: mean time (ms), std deviation, and throughput
    (images/sec)
  4. Accepts configurable image sizes via CLI args: 256x256, 512x512, 1024x1024

Expected output format:

Operation kornia-rs (ms) OpenCV (ms) Speedup
resize 512x512 0.31 0.45 1.45x
rgb_to_grayscale 0.12 0.18 1.50x
gaussian_blur 0.87 1.10 1.26x

πŸ“š Library Reference

  • OpenCV Python (cv2) β€” used as the performance baseline for all imgproc ops
  • pytest-benchmark β€” https://pytest-benchmark.readthedocs.io/en/latest/
  • Criterion.rs (existing Rust benchmarks in kornia-rs) β€” the Python script
    mirrors what Criterion already does at the Rust level, but exposes it
    through the PyO3 Python bindings layer
  • NumPy β€” for generating consistent test input arrays across both libraries

πŸ”„ Alternatives Considered

  • Extending existing Criterion.rs benchmarks: Criterion benchmarks only run
    at the Rust level and are not accessible via Python bindings. Users working
    through the Python API have no equivalent tool.

  • Using timeit directly in a notebook: viable but not reproducible or
    CI-friendly. A pytest-benchmark script can be integrated into the CI
    pipeline and tracked over time.

🎯 Use Cases

  1. GSoC 2026 contributors working on the ONNX/TensorRT backend project need
    a baseline benchmark to quantify GPU speedups β€” this script provides that
    reference point.

  2. Users evaluating kornia-rs vs OpenCV for production pipelines can run
    this script locally to make an informed decision based on their hardware.

  3. Maintainers can integrate it into CI to catch performance regressions
    across releases β€” complementing the benchmark README results in publish benchmark results in README or docsΒ #718.

πŸ“ Additional Context

This issue is raised as part of pre-application engagement for GSoC 2026,
specifically targeting the "Expand kornia-vlm with ONNX/TensorRT backends"
project. The benchmarking infrastructure proposed here directly supports
that project's deliverable of comparing CPU vs CUDA vs TensorRT inference.

I am happy to implement this as my pre-application contribution if a
maintainer confirms the scope and preferred tooling (pytest-benchmark
vs timeit vs custom harness).

Related: #718 (publish benchmark results in README/docs)

🀝 Contribution Intent

  • I plan to submit a PR to implement this feature
  • I'm requesting this feature but not planning to implement it

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is neededtriagewait for a maintainer to approve and assign this ticket

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions