🧩 Operator Coverage Checker for ONNX Models

A lightweight tool to check if an ONNX model is compatible with a given accelerator by validating its operators (ops) against a supported-ops list.

This README includes the sections you asked for: Usage, Visualization, Project Structure, Demo Graph, and Summary — all in Markdown and ready to copy.

---

Usage

Installation

git clone https://github.com/Dhairya-Dudhatra/operator-coverage-checker.git
cd operator-coverage-checker
pip install -r requirements.txt

1) Export a HuggingFace model to ONNX

python examples/export_model.py
# → saves model at ./distilbert.onnx

2) Run the operator coverage checker (CLI)

python main.py examples/distilbert.onnx --ops coverage_checker/supported_ops.json

The CLI prints a concise table listing operators, counts, and whether each is supported.
Example output:

Operator     Count    Supported
-----------  -------  ----------
Add          12       ✅
Gelu          6       ❌
LayerNorm     2       ❌
MatMul       10       ✅
Relu          5       ✅

3) Run the checker + generate visualization (CLI)

python main.py examples/distilbert.onnx --ops coverage_checker/supported_ops.json --visualize
# By default, saves image at: docs/sample_graph.png

Change output path:

python main.py examples/distilbert.onnx --ops coverage_checker/supported_ops.json --visualize --output docs/my_graph.png

---

Visualization

The project provides both a CLI flag (--visualize) and a Python API.

Recommended flow: run the checker to identify unsupported ops, then either use the CLI --visualize flag (one-liner) or call the visualization helper from Python if you want programmatic control.

Python API example

from coverage_checker.visualizer import visualize_model

# If you already know unsupported ops (or got them from the CLI output)
unsupported_ops = {"Gelu", "LayerNorm"}

# Default saves at docs/sample_graph.png
visualize_model("examples/distilbert.onnx", unsupported_ops)

# Or specify output explicitly
visualize_model("examples/distilbert.onnx", unsupported_ops, output_path="docs/sample_graph.png")

What the visualization does:

• Builds a simplified op-level graph (not every tensor edge).
• Colors nodes: green = supported, red = unsupported.
• Saves a PNG image to docs/sample_graph.png (or the path you pass).

Notes / troubleshooting

• Ensure networkx and matplotlib are installed (in requirements.txt).
• The visualizer intentionally simplifies the graph (collapses repeated op types) to avoid unreadable hairballs.
• If the image is blank or extremely dense, try running the checker and passing a smaller unsupported-ops set for testing.

---

Project Structure

operator-coverage-checker/
├── coverage_checker/
│   ├── __init__.py
│   ├── checker.py          # Core logic: load ONNX, extract ops, compare against supported list
│   ├── visualizer.py       # Graph visualization utility (saves PNG)
│   └── supported_ops.json  # Example accelerator ops list (JSON)
├── examples/
│   ├── export_model.py     # Export HuggingFace model to ONNX
│   └── distilbert.onnx     # Example exported model (gitignore or download)
├── docs/
│   └── sample_graph.png    # Visualization output (generated by visualizer)
├── main.py                 # CLI entrypoint (checker + --visualize)
├── requirements.txt
└── README.md

---

Demo Graph

Here is the place where the generated visualization is referenced in the README:

(If docs/sample_graph.png does not exist yet, run python main.py examples/distilbert.onnx --ops coverage_checker/supported_ops.json --visualize to create it.)

---

Summary

• Install dependencies from requirements.txt.
• Export an ONNX model with examples/export_model.py.
• Run the operator coverage check:
  • python main.py examples/distilbert.onnx --ops coverage_checker/supported_ops.json
• Generate a visualization with:
  • CLI: --visualize (saves to docs/sample_graph.png by default)
  • Python: from coverage_checker.visualizer import visualize_model
• Output locations: default image path is docs/sample_graph.png (override with --output).